You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "voonhous (via GitHub)" <gi...@apache.org> on 2025/04/16 09:56:35 UTC
[PR] [HUDI-9315][RFC-94] Add RFC-94: Hudi Timeline UI [hudi]
voonhous opened a new pull request, #13152:
URL: https://github.com/apache/hudi/pull/13152
### Change Logs
Add RFC-94.
### Impact
None
### Risk level (write none, low medium or high below)
None
### Documentation Update
None
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4943193414
@wombatu-kun All comments addressed :) Thank you for the review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3565992083
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+Both `getTableConfig` and `getSchemaHistory` reuse a `HoodieTableMetaClient` cached per basepath in a
+`ConcurrentHashMap`, so repeated requests skip full metaClient construction. A `HoodieTableMetaClient` snapshots its
+`tableConfig` and `activeTimeline` at construction and refreshes them only on demand, so a build-once cache would serve
+stale config/schema for the lifetime of a long-lived embedded driver. To avoid that, each config/schema request first
+reloads the relevant snapshot under a per-basepath guard - `reloadTableConfig()` for `getTableConfig`,
Review Comment:
Fair flag. One correction: the accessors aren't unguarded -- `HoodieTableMetaClient.getActiveTimeline()`, `reloadActiveTimeline()` and `reloadTableConfig()` are all `synchronized` (HoodieTableMetaClient.java:512/541/549), and the field swaps are atomic reference assignments, so there's no torn-object read. The real gap is the one you point at: a compound `reload()` then read isn't a single critical section, so a concurrent same-basepath reload can swap the snapshot between one request's reload and its read.
Rather than wrap reload+read in a per-basepath lock, I took your second option and dropped the shared cache: each request now builds a short-lived `HoodieTableMetaClient` for its basepath, reads, and discards it. No shared mutable state, no lock, and it's equally cheap at click frequency since the data was being re-read on every request anyway (the cache only ever saved the metaClient scaffolding, not the IO). Updated the handler, architecture, feature and multi-table sections to match.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3565947310
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
Review Comment:
Confirmed and fixed -- thanks, this was a real bug in the design. Traced it: `getTimeline()` -> `visibleCommitsAndCompactionTimeline` = `visibleActiveTimeline.getWriteTimeline()`, and `getWriteTimeline()` keeps only `{commit, deltacommit, compaction, logcompaction, replacecommit, clustering}` (BaseHoodieTimeline.java:196-200); the view's `visibleActiveTimeline` is `metaClient.getActiveTimeline().filterCompletedAndCompactionInstants()` (FileSystemViewManager.java:146/163/175). So it can't surface clean/rollback/savepoint/restore/indexing or any requested/inflight state.
Switched `getTimelineV2` to the per-basepath `HoodieTableMetaClient`'s `getActiveTimeline()` (already used for config/schema), which carries all 11 actions in every state. Also reconciled the ripples so the RFC stays consistent:
- Architecture diagram: `API --> FSVM --> Meta` becomes `API --> Meta` (`HoodieTableMetaClient`).
- REST-request description and the Backend intro now name the metaClient as the timeline source.
- Freshness paragraph: `getTimelineV2` joins the per-request `reloadActiveTimeline()` path.
- Dropped the stale "timeline view reads the server-synced FileSystemView" line.
Left a short note at the handler method spelling out why the `FileSystemView` timeline is unusable, so this doesn't get reintroduced.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3568035417
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. The instant is built with the request's metaClient via `metaClient.getInstantGenerator()`;
+ a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+`getTimelineV2`, `getTableConfig` and `getSchemaHistory` each build a short-lived `HoodieTableMetaClient` for the
Review Comment:
Right, that was an oversight -- `getInstantDetails` builds its instant from the request's metaClient and reads through the active timeline, so it belongs in both lists.
Fixed: the paragraph now names all four methods and states the cost per route rather than lumping it:
- every route pays one `hoodie.properties` read;
- all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to map it, `getInstantDetails` to resolve the instant it reads via `getContentStream`, `getSchemaHistory` to walk the completed commits);
- `getInstantDetails` then reads a single instant file's content on top of that.
Please take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315][RFC-94] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3401530297
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
+The vis-timeline library is loaded with a two-tier strategy:
+
+1. **Primary:** Load from the `unpkg.com` CDN for automatic patch updates.
+2. **Fallback:** If the CDN is unreachable (e.g., air-gapped environments), load from the bundled
+ copy at `/lib/vis-timeline/`.
+
+The bundled fallback adds ~300KB to the JAR but ensures the UI works without internet access.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`,
+ `clustering`, `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These
+ correspond to the actions in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to
+ `completionTime`. Non-completed instants (requested or inflight) are rendered as point items at
+ `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/timeline/instant/details` and
+ shows the deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths
+since the `FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|--------------------------------|----------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------------------|
+| GET | `/v2/timeline/instants` | `basepath` (required) | `List<InstantDTO>` (v2) | Returns all active instants with requested time, completion time, action, and state |
+| GET | `/v2/timeline/instant/details` | `basepath` (required), `instantTime` (required), `action` (required) | JSON string | Returns the deserialized content of a specific instant's metadata (Avro -> JSON) |
+
+Static files are served from the `/ui/` path, mapped to classpath resources under
+`src/main/resources/public/`.
+
+#### DTO Design
+
+A new v2 `InstantDTO` is introduced in a `v2` package to avoid modifying the existing `/v1/` API
+contract. The existing `InstantDTO` (in `o.a.h.common.table.timeline.dto`) only exposes `action`,
+`timestamp` (requested time), and `state` - it lacks `completionTime`, which is needed for
+rendering range bars on the timeline.
+
+The v2 `InstantDTO` contains:
+
+- `requestedTime` - the instant's requested timestamp (`HoodieInstant.requestedTime()`)
+- `completionTime` - the instant's completion timestamp (`HoodieInstant.getCompletionTime()`), null
+ for non-completed instants
+- `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+- `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+#### Handler Design
+
+A new `UiHandler` class extends `Handler` (following the `InstantStateHandler` pattern) with two
+methods:
+
+1. `getActiveInstants(basePath)` - Retrieves the active timeline via
+ `viewManager.getFileSystemView(basePath).getTimeline()`, iterates over all instants, and maps
+ each `HoodieInstant` to a v2 `InstantDTO` including completion time.
+
+2. `getInstantDetails(basePath, instantTime, action)` - Constructs a `HoodieTableMetaClient` for
+ the given basepath, reads the instant's Avro content via `ActiveTimeline.getInstantDetails()`,
+ and deserializes it to JSON for display in the UI.
+
+#### Registration in RequestHandler
+
+The `UiHandler` is registered in `RequestHandler` following the existing pattern:
+
+- A `UiHandler` field is added alongside the existing handler fields.
+- A `registerUiAPI()` method is added and called from `register()`.
+- Registration is gated behind an `--enable-ui` config flag, same as `enableMarkerRequests` and
+ `enableInstantStateRequests`.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table
+ path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The main feature for the first cut is timeline visualization for a Hudi table.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are
+ rendered
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated.
+Completed instants appear as horizontal bars whose width represents duration (requested ->
+completed). Inflight and requested instants appear as point markers. Color indicates state: green
+for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui/`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`).
+The `--enable-ui` flag controls whether the UI static files and `/v2/timeline/` API endpoints are
+registered. When the flag is not set, the timeline server behaves exactly as it does today -
+no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a
+timeline server within the driver process. The UI can be enabled on this embedded server by setting
+a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the
+same UI routes and static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui/`. The embedded server port is logged at startup
+and can also be retrieved from the Spark configuration via the
+`hoodie.embed.timeline.server.port` property.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write
+client initializes and stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a
+"Hudi Timeline" tab is registered in the Spark web UI (typically at `http://localhost:4040`). This
+gives users a single place to discover and access the Hudi Timeline UI without needing to know the
+embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab
+contains a single `WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded
+timeline server at `http://<driver-host>:<timeline-server-port>/ui/`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues
+within the Spark UI shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab,
+providing the complete interactive experience.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration happens in `EmbeddedTimelineService` after the embedded server has started
+successfully. The tab is detached during shutdown to ensure clean cleanup. If the Spark UI is not
+available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement
+
+The Spark UI tab implementation lives in `hudi-spark-client`, not `hudi-timeline-service`, because
+it depends on Spark APIs (`WebUITab`, `WebUIPage`) which are not available in the timeline service
+module. The `EmbeddedTimelineService` (also in `hudi-client-common`) coordinates the registration
+by calling into the Spark-specific tab class when running with a `HoodieSparkEngineContext`.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file
+ serving; no template engine is added.
+- **vis-timeline JS/CSS:** ~300KB bundled as static resources under
+ `src/main/resources/public/lib/vis-timeline/`.
+- **Spark UI tab:** Uses existing `spark-core` APIs (`WebUITab`, `WebUIPage`) which are already
+ provided dependencies in `hudi-spark-client`. No new JARs are added.
+- **No impact on Spark/Flink bundles.** `hudi-timeline-server-bundle` is a separate artifact;
+ adding static resources does not affect engine-specific bundles.
+- **No frontend build pipeline.** No npm, webpack, or vite. The JS/CSS files are committed directly
+ and served as-is.
+
+## Additional Quality of Life (QoL) Features to be Considered
+
+The following client-side enhancements were implemented alongside the core features above. These are
+purely frontend additions - no new APIs, Java dependencies, or architectural changes beyond what is
+specified in this RFC.
+
+- **Bootstrap 5 + renderjson styling** - Bootstrap 5 for layout and renderjson for collapsible,
+ syntax-highlighted JSON in the detail panel.
+- **Instant search and "Go to Now" button** - A search input that filters and focuses the timeline
+ on a specific instant by requested time, plus a button to jump to the latest instant.
+- **Summary statistics bar** - Counts of instants by state (completed, inflight, requested)
+ computed client-side from the loaded data.
+- **Filter controls (state + action)** - Dropdown filters for state and action type using
+ vis-timeline's `DataView` filtering. Users can isolate specific instant types.
+- **Color legend** - Maps colors to instant states for quick reference.
+- **URL `?path=` persistence** - The basepath is stored in the URL query parameter so the view can
+ be bookmarked and shared.
+- **Keyboard navigation** - Left/right arrow keys to move between instants, Escape to close the
+ detail panel.
+- **Help modal** - Pressing `?` opens a modal listing all keyboard shortcuts and UI interactions.
+
+## Future Improvements
+
+Other features we can add later:
+
+1. **Incremental clean range overlay** - A translucent overlay from `lastCompletedCommitTimestamp`
+ to `earliestCommitToRetain` when clean metadata is available, showing which instants are
+ protected from cleaning and which are eligible for removal.
+
+2. **Partition/filegroup write heatmap** - Within one or multiple instants, render a grid colored
+ by write volume from `HoodieCommitMetadata.partitionToWriteStats`. Helps identify skewed
+ partitions or heavily-modified filegroups across commits.
+
+3. **Metadata table timeline overlay** - Render the metadata table's timeline alongside the main
+ table timeline. Useful for diagnosing cases where the metadata table is lagging or has stale
+ entries.
+
+4. **Archived timeline** - Load and render instants from the LSM-based archive files, extending
+ the view beyond the active timeline to include historical actions.
+
+5. **Per-state instant details** - Currently the plan for the UI is to only render the highest state
+ for each instant (e.g., if requested, inflight, and completed all exist, only completed is shown).
+ We should keep this as the default but also let users view the other states' metadata.
+ The simplest approach would be adding state toggle buttons after selecting an instant, so users
+ can fetch and view the JSON content for each state individually.
+
+## Rollout/Adoption Plan
+
+- **Backward compatibility:** Additive change only. All existing `/v1/` endpoints remain unchanged.
+ UI endpoints are under `/v2/timeline/` and static files at `/ui/`. No existing behavior changes.
+- **Release target:** 1.2.x
+- **Documentation:** A new page on hudi.apache.org under the "Operations" section with screenshots,
+ a startup guide, and a debugging walkthrough showing how to use the timeline UI to diagnose common
+ issues (stuck compactions, long-running commits, etc.).
+- **No migration needed:** Net-new feature behind an opt-in flag (`--enable-ui`), no impact on
+ existing users or running systems.
+
+## Test Plan
+
+### Unit Tests
+
+- **`UiHandler.getActiveInstants()`**: Verify correct mapping from `HoodieInstant` to v2
+ `InstantDTO`, including:
+ - All action types in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` are mapped correctly.
+ - `completionTime` is populated for completed instants and null for requested/inflight instants.
+ - Instants are returned in timeline order.
+- **`UiHandler.getInstantDetails()`**: Verify correct Avro deserialization to JSON for various
+ action types (commit metadata, compaction plan, clean plan, etc.).
+- **Error cases**: Invalid basepath returns an appropriate error. Empty timeline returns an empty
+ list.
+
+### Integration Tests
+
Review Comment:
Good suggestion. Added an "Alternatives Considered" section covering: standalone React/Node SPA (rejected - adds a frontend build pipeline and JS dependency tree to a Java project; static vanilla JS needs no build and no new Java deps), richer CLI output (rejected - text can't convey concurrency/overlap/duration, which is the point), a new standalone UI service/module (rejected - the timeline server already serves per-basepath metadata standalone and embedded), server-side templating (rejected - unnecessary Java dependency), and iframe embedding in the Spark UI (rejected for layout/scroll issues, hence the link approach).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3565960613
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
Review Comment:
Both correct, fixed in item 2 of the handler design:
- `getInstantDetails()` -> `getContentStream(instant)`. Confirmed the former is `@Deprecated` on `HoodieInstantReader` (it just delegates to `getContentStream`), so the new `/v2/` endpoint now specs the non-deprecated method.
- "the timeline's own layout-aware `InstantGenerator`" -> `metaClient.getInstantGenerator()`. Confirmed `BaseHoodieTimeline.instantGenerator` is `protected` with no accessor; the handler already holds the metaClient, so its public `getInstantGenerator()` is the natural source.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3569656367
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,879 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as. Equal to `action` for everything
+ except pending `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it
+ server-side keeps the mapping table in one place rather than duplicating it in JavaScript.
+
+ Concretely, this is `InstantComparatorV2.getComparableAction(action)`. Note there is no version-dispatched
+ accessor to call instead: `getComparableAction` is a `public static` on `InstantComparatorV1`/`V2`, the
+ `InstantComparator` interface declares only the three comparator getters, and the one place the mapper is
+ selected by layout version (`TimelineLayout.filterHoodieInstantsByLatestState`) is private.
+
+ Pinning the v2 static is safe for layout-v1 tables because the two maps are *equivalent over everything a v1
+ timeline can emit*. They agree on `compaction -> commit` and `logcompaction -> deltacommit`. The only difference
+ is v2's `clustering -> replacecommit` entry, and v1 has no `clustering` action to apply it to: in 0.x, clustering
+ and replace-commit shared a filename (`InstantFileNameGeneratorV1.makeRequestedClusteringFileName()` delegates to
+ `makeRequestedReplaceFileName()`), so a v1 clustering instant is written as `.replacecommit.requested` and already
+ carries `action=replacecommit`. The mapping is a no-op there, which is exactly why `InstantComparatorV1`'s map has
+ no `clustering` entry - there is nothing to map, not something missing.
+
+ If a second caller ever needs this mapping, lifting `getComparableAction` onto `InstantComparator` (or
+ `TimelineLayout`) is the clean follow-up; doing it here would mean changing a shared `hudi-common` interface for a
+ UI display field, which this RFC does not need.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` via `InstantComparatorV2.getComparableAction(action)` - the same mapping
+ the filter used (see [DTO Design](#dto-design)) - and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ **Stream ownership applies to every timeline content read in this RFC, not just this one.** Unlike the deprecated
+ `getInstantDetails()`, which read into a `byte[]` and closed the stream itself, `getContentStream` hands back a raw
+ open stream (`metaClient.getStorage().open(...)`) that the caller owns. Every such read - this route, and the up-to
+ `limit` per-instant commit-metadata reads in [Schema-History Reconstruction](#schema-history-reconstruction) - goes
+ through a single shared helper that closes under try-with-resources, so the handle is released on the
+ deserialization-failure path and not only on success. The schema-history path is the one that actually matters: at up
+ to 1000 reads per request, with a Refresh control re-issuing the fetch, a leaked handle there exhausts file
+ descriptors quickly.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema, resolved with
+ `TableSchemaResolver.getTableSchemaIfPresent(metaClient.getTableConfig().populateMetaFields())`; `null` when the
+ `Option` is empty.
+
+ Not `getTableSchema()`: that overload ends in `getTableSchemaInternal(...).orElseThrow(schemaNotFoundError())` and
+ raises `HoodieSchemaNotFoundException` on a table with no resolvable schema, so it cannot express "unresolved" as a
+ return value at all. `getTableSchemaIfPresent(boolean)` returns `Option<HoodieSchema>` and does not throw, which is
+ what this view needs - a brand-new table with no commits is a normal thing to point the UI at, not a 500. There is no
+ no-arg `getTableSchemaIfPresent`, so the call passes `populateMetaFields()` explicitly, which is exactly what the
+ no-arg `getTableSchema()` was supplying on our behalf.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
Review Comment:
`history` scans only the most recent `limit` completed commits and the response carries no marker for that window, so a table with more commits than `limit` returns a silently truncated history whose oldest entry is emitted as a schema change even when the schema was stable well before it. Consider returning the scanned window (oldest instant scanned, and whether older commits exist) so the UI can label that first entry as a baseline rather than a change.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,879 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
Review Comment:
`/v1/hoodie/view` is RemoteHoodieTableFileSystemView's RPC namespace and it already defines `timeline/instants/all`, so `/v2/hoodie/view/*` reads as a v2 of that protocol rather than a UI API, and it consumes the URL space a real FSV v2 would want. The RFC already namespaces the assets under `/ui` for exactly this reason - was leaving the four API routes outside `/ui` (rather than `/ui/api/*`) deliberate?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3566195312
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
Review Comment:
Neither view is `/v1`-parity after the switch to `metaClient.getActiveTimeline()`.
The timeline view is now a superset: v1's `timeline/instants/all` is served from the `FileSystemView`'s write timeline, filtered to completed plus (log)compaction instants, while `/v2` reads the full active timeline and so additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants and every requested/inflight state.
The instant-detail view has no `/v1` counterpart at all: no route on `RemoteHoodieTableFileSystemView` returns instant content. `/v2/hoodie/view/timeline/instant` returns a deserialized `HoodieCommitMetadata`, which carries per-partition write stats and the table schema under the `schema` key of `extraMetadata`, the same schema exposure this paragraph counts as widening when it comes from the schema-history view.
List both alongside table-config and schema-history rather than claiming parity for them.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
Review Comment:
The active timeline collapses states before the handler sees them. `HoodieTableMetaClient.getActiveTimeline()` builds an `ActiveTimelineV2` with `applyLayoutFilter` true, which runs `TimelineLayout.filterHoodieInstantsByLatestState`: instants are grouped by `(requestedTime, getComparableAction(action))` and only the highest state survives. `InstantComparatorV2.COMPARABLE_ACTIONS` maps `compaction` to `commit`, `logcompaction` to `deltacommit`, and `clustering` to `replacecommit`.
So a completed compaction reaches `getTimelineV2` as a single instant whose action is `commit`, since its completed file is `.commit`. The `compaction`, `logcompaction` and `clustering` rows can then only ever hold pending point items, and a completed compaction renders as a range bar in the `commit` row, indistinguishable from an ordinary commit. That undercuts the stuck-compaction diagnosis this RFC opens with, and it also makes the Test Plan's "all action types in `VALID_ACTIONS_IN_TIMELINE` are mapped correctly" unsatisfiable as written.
Specify how the UI recovers the originating action, for example by carrying the pending instant's action on `InstantDTOV2`, or by grouping on the requested/inflight file rather than on the surviving instant.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. The instant is built with the request's metaClient via `metaClient.getInstantGenerator()`;
+ a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+`getTimelineV2`, `getTableConfig` and `getSchemaHistory` each build a short-lived `HoodieTableMetaClient` for the
Review Comment:
`getInstantDetails` is missing from this list, yet item 2 above has it building its instant from "the request's metaClient" and reading through the active timeline's `getContentStream`, so it needs one too. It also pays the active-timeline listing that the cost sentence attributes only to the timeline and schema routes.
Add it to both the enumeration and the cost model, or state where `getInstantDetails` gets its metaClient.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. The instant is built with the request's metaClient via `metaClient.getInstantGenerator()`;
+ a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+`getTimelineV2`, `getTableConfig` and `getSchemaHistory` each build a short-lived `HoodieTableMetaClient` for the
+request's basepath, read from it, and discard it - no metaClient is shared across Javalin's request threads. At
+human-click frequency the construction cost (one `hoodie.properties` read, plus one active-timeline listing for the
+timeline and schema routes) is negligible, and a fresh per-request instance is always current and keeps each request's
+read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reads the table config / active timeline fresh on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server binds a free port by default
+(`hoodie.embed.timeline.server.port` = `0`), and the bound port is not written back to that property - read it from the
+startup log line (`Started embedded timeline server at <host>:<port>`), or from the resolved
+`FileSystemViewStorageConfig` remote-server port that `EmbeddedTimelineService` propagates it to.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, and each UI request builds a `HoodieTableMetaClient` for its
+own basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **LICENSE/NOTICE obligations.** Each vendored library needs a "This product bundles ..." stanza in the source-release
+ top-level `LICENSE` (and in the `hudi-timeline-server-bundle` LICENSE, since the assets ship inside that JAR) naming
+ the library, its license, and its copyright, with the full MIT/ISC license text inlined the same way existing bundled
+ code is handled in `LICENSE`. The minified files already carry their upstream copyright headers, which must be
+ preserved. No `NOTICE` changes are required: MIT and ISC do not mandate NOTICE entries, and vis-timeline is taken
+ under its MIT option (ASF policy discourages adding MIT/ISC copyrights to `NOTICE`); were vis-timeline instead taken
+ under Apache-2.0, any upstream `NOTICE` content it ships would have to be propagated.
+- **Spark UI tab (planned follow-up):** Will use existing `spark-core` APIs (`WebUITab`, `WebUIPage`), already provided
+ in `hudi-spark-client`. No new JARs are added.
+- **No impact on Spark/Flink bundles.** `hudi-timeline-server-bundle` is a separate artifact; adding static resources
Review Comment:
`hudi-timeline-service` is a compile-scope dependency of `hudi-client-common`, and `hudi-spark-bundle`, `hudi-flink-bundle`, `hudi-utilities-bundle` and `hudi-kafka-connect-bundle` all list `org.apache.hudi:hudi-timeline-service` in their shade `artifactSet`. Their `<filters>` exclude only signature files, `META-INF/services/javax.*` and `**/*.proto`, so everything under `src/main/resources/public/` is unpacked into each of those JARs. The module has no `src/main/resources` today, so the ~890KB of vendored assets is net-new payload in every engine bundle, not only `hudi-timeline-server-bundle`.
Restate this bullet accordingly, and widen the LICENSE/NOTICE bullet above to cover those bundles. If the assets are meant to stay out of the engine bundles, that needs an explicit shade filter excluding `public/**` in each bundle pom, which is worth specifying here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3568937163
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
Review Comment:
There is no `getComparableAction` on `InstantComparator`; the interface declares only `actionOnlyComparator`, `requestedTimeOrderedComparator` and `completionTimeOrderedComparator`. `getComparableAction` is a `public static` on `InstantComparatorV1` and `InstantComparatorV2`, and Java statics are not reachable through an interface-typed reference, so "the table's `InstantComparator`" has no such method to call.
Nor is there a version-dispatched public accessor: the only place the mapper is selected by layout version is inside `TimelineLayout.filterHoodieInstants`, whose `filterHoodieInstantsByLatestState` is private, and `InstantComparators.ActionComparator` keeps its own copy private over an injected map.
Name the source concretely - either `InstantComparatorV2.getComparableAction(action)` (safe in practice: `InstantComparatorV1`'s map has no clustering entry, but `ActiveTimelineV1`'s valid extensions exclude clustering too, so a layout-v1 timeline can never surface an instant the two maps disagree on), or propose lifting `getComparableAction` onto `InstantComparator` / `TimelineLayout` as part of this work.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
Review Comment:
`TableSchemaResolver.getTableSchema()` cannot return null: it delegates to `getTableSchema(boolean)`, which ends in `getTableSchemaInternal(...).orElseThrow(schemaNotFoundError())` and raises `HoodieSchemaNotFoundException`. On a table with no commits it throws, which contradicts both the "no error" claim in the cost model below and the empty-timeline behaviour in Error Handling.
`getTableSchemaIfPresent(boolean)` on the same class returns `Option<HoodieSchema>` and does not throw - that is the API that matches what this bullet describes.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reads the table config / active timeline fresh on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server binds a free port by default
+(`hoodie.embed.timeline.server.port` = `0`), and the bound port is not written back to that property - read it from the
+startup log line (`Started embedded timeline server at <host>:<port>`), or from the resolved
+`FileSystemViewStorageConfig` remote-server port that `EmbeddedTimelineService` propagates it to.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, and each UI request builds a `HoodieTableMetaClient` for its
+own basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **The assets must be explicitly excluded from the engine bundles.** This does *not* come for free.
+ `hudi-timeline-service` is a compile-scope dependency of `hudi-client-common`, and six bundles list
+ `org.apache.hudi:hudi-timeline-service` in their shade `artifactSet`: `hudi-timeline-server-bundle` (intended) plus
+ `hudi-spark-bundle`, `hudi-flink-bundle`, `hudi-utilities-bundle`, `hudi-kafka-connect-bundle` and
+ `hudi-integ-test-bundle`. Their `<filters>` currently exclude only signature files, `META-INF/services/javax.*` and
+ `**/*.proto`, so *any* resource under `src/main/resources/` is unpacked into each shaded JAR. The module has no
+ `src/main/resources` today, which is exactly why this has never bitten us - the ~890KB of vendored assets would be
+ net-new payload in all six bundles, not just the server bundle.
+
+ The first cut is standalone-only, so the UI cannot even be served from an embedded timeline server inside a Spark or
+ Flink driver. Shipping ~890KB of unreachable JS/CSS in five engine bundles is pure bloat, and it would drag the
+ LICENSE obligations below into each of them. The five non-server bundle poms therefore add an explicit shade filter:
+
+ ```xml
+ <filter>
+ <artifact>org.apache.hudi:hudi-timeline-service</artifact>
+ <excludes>
+ <exclude>public/**</exclude>
+ </excludes>
+ </filter>
+ ```
+
+ Only `hudi-timeline-server-bundle` ships the assets. When the embedded/Spark-UI follow-up lands, it must lift this
+ exclusion for the bundles that actually gain an embedded UI (`hudi-spark-bundle`, and `hudi-flink-bundle` if it
+ follows), and add the LICENSE stanzas below to those bundles at the same time. That is a deliberate, reviewable step
+ in the follow-up, not something to inherit silently here.
+- **LICENSE/NOTICE obligations.** Each vendored library needs a "This product bundles ..." stanza in the source-release
+ top-level `LICENSE` **and** in the LICENSE of every bundle whose JAR actually contains the assets, naming the library,
+ its license, and its copyright, with the full MIT/ISC license text inlined the same way existing bundled code is
+ handled in `LICENSE`. With the shade filter above, that set is exactly the source release plus
+ `hudi-timeline-server-bundle`; without it, it would be all six bundles. The minified files already carry their
+ upstream copyright headers, which must be preserved. No `NOTICE` changes are required: MIT and ISC do not mandate
+ NOTICE entries, and vis-timeline is taken under its MIT option (ASF policy discourages adding MIT/ISC copyrights to
+ `NOTICE`); were vis-timeline instead taken under Apache-2.0, any upstream `NOTICE` content it ships would have to be
+ propagated.
+- **Spark UI tab (planned follow-up):** Will use existing `spark-core` APIs (`WebUITab`, `WebUIPage`), already provided
Review Comment:
`WebUITab`, `WebUIPage`, `WebUI.attachTab`/`detachTab` and `SparkContext.ui` are all `private[spark]` in Spark's `WebUI.scala` / `SparkContext.scala` - Spark internals, not public spark-core API. Two consequences the design does not account for.
Scala code outside an `org.apache.spark` package cannot reference them at all, so the tab has to be declared in an `org.apache.spark` package; the repo already does this for `HoodieSparkKryoRegistrar`, under `hudi-spark-client`'s `src/main/scala/org/apache/spark/`. Java code can reach them, since Scala's qualified-private erases to bytecode-public - but `WebUIPage.render` returns `Seq[Node]`, which erases to `scala.collection.Seq` in `spark-core_2.12` and `scala.collection.immutable.Seq` in `spark-core_2.13`, so a single Java source cannot implement it for both Scala profiles Hudi cross-builds.
Worth stating here: the tab is Scala, lives in an `org.apache.spark` package, and rests on Spark internals that carry no cross-version stability guarantee over the Spark 3.3-4.0 range Hudi supports.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
Review Comment:
`getHistorySchemaStr()` is not "one `.schema` history-file read". It delegates to `getHistorySchemaStrByGivenValidCommits(EMPTY_LIST)`, which calls `getValidInstants()` - a walk of `getCommitsTimeline().filterCompletedInstants()` - before it checks `storage.exists(baseSchemaPath)`. So the completed-commits timeline is walked on every schema-history request even for a table with no `.schema` directory, which is exactly the "no extra scanning" case claimed just below.
The constructor matters too: `FileBasedInternalSchemaStorageManager(HoodieStorage, StoragePath)` lazily builds a second `HoodieTableMetaClient` inside `getMetaClient()`, so each request would pay a second `hoodie.properties` read and a second timeline listing. Pass the request's metaClient to the `FileBasedInternalSchemaStorageManager(HoodieTableMetaClient)` overload, and either check for `.schema` in the handler before calling or drop the "no extra scanning" claim.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reads the table config / active timeline fresh on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server binds a free port by default
+(`hoodie.embed.timeline.server.port` = `0`), and the bound port is not written back to that property - read it from the
+startup log line (`Started embedded timeline server at <host>:<port>`), or from the resolved
+`FileSystemViewStorageConfig` remote-server port that `EmbeddedTimelineService` propagates it to.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, and each UI request builds a `HoodieTableMetaClient` for its
+own basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **The assets must be explicitly excluded from the engine bundles.** This does *not* come for free.
+ `hudi-timeline-service` is a compile-scope dependency of `hudi-client-common`, and six bundles list
+ `org.apache.hudi:hudi-timeline-service` in their shade `artifactSet`: `hudi-timeline-server-bundle` (intended) plus
+ `hudi-spark-bundle`, `hudi-flink-bundle`, `hudi-utilities-bundle`, `hudi-kafka-connect-bundle` and
+ `hudi-integ-test-bundle`. Their `<filters>` currently exclude only signature files, `META-INF/services/javax.*` and
+ `**/*.proto`, so *any* resource under `src/main/resources/` is unpacked into each shaded JAR. The module has no
+ `src/main/resources` today, which is exactly why this has never bitten us - the ~890KB of vendored assets would be
+ net-new payload in all six bundles, not just the server bundle.
+
+ The first cut is standalone-only, so the UI cannot even be served from an embedded timeline server inside a Spark or
+ Flink driver. Shipping ~890KB of unreachable JS/CSS in five engine bundles is pure bloat, and it would drag the
+ LICENSE obligations below into each of them. The five non-server bundle poms therefore add an explicit shade filter:
+
+ ```xml
+ <filter>
+ <artifact>org.apache.hudi:hudi-timeline-service</artifact>
+ <excludes>
+ <exclude>public/**</exclude>
+ </excludes>
+ </filter>
+ ```
+
+ Only `hudi-timeline-server-bundle` ships the assets. When the embedded/Spark-UI follow-up lands, it must lift this
Review Comment:
Stripping `public/**` does not stop `--enable-ui` from being reachable in those five bundles: each of the six lists `org.apache.hudi:hudi-timeline-service` in its shade `artifactSet`, and the filter drops only resources, so `TimelineService` and its `main()` still ship in all six.
Enabling the UI from a stripped bundle then fails in a misleading way. Javalin 4.6.7 resolves a `Location.CLASSPATH` static-file directory eagerly, in `ConfigurableHandler`'s constructor via `getResourceBase`, throwing `JavalinException("Static resource directory with path: '/public' does not exist.")` at `Javalin.create()` - i.e. inside `TimelineService.createApp()`. `startServiceOnPort` catches every `Exception`, and since this is not a `JavalinBindException` it logs "Timeline server start failed on port N. Attempting port N + 1", retries `START_SERVICE_MAX_RETRIES` (16) times, and finally throws `IOException("Timeline server start failed on port N, after retry 16 times")`. A missing classpath resource surfaces as a port failure.
Once the assets and the flag are decoupled per bundle, the flag can no longer be the only gate. Specify that static-file registration is conditional on the assets being present (`TimelineService.class.getResource("/public") != null`), failing fast with an explicit "UI assets not bundled in this jar" message. The embedded follow-up sharpens this rather than removing it, since it lifts the exclusion for some bundles and not others.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reads the table config / active timeline fresh on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server binds a free port by default
+(`hoodie.embed.timeline.server.port` = `0`), and the bound port is not written back to that property - read it from the
+startup log line (`Started embedded timeline server at <host>:<port>`), or from the resolved
+`FileSystemViewStorageConfig` remote-server port that `EmbeddedTimelineService` propagates it to.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, and each UI request builds a `HoodieTableMetaClient` for its
+own basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **The assets must be explicitly excluded from the engine bundles.** This does *not* come for free.
+ `hudi-timeline-service` is a compile-scope dependency of `hudi-client-common`, and six bundles list
+ `org.apache.hudi:hudi-timeline-service` in their shade `artifactSet`: `hudi-timeline-server-bundle` (intended) plus
+ `hudi-spark-bundle`, `hudi-flink-bundle`, `hudi-utilities-bundle`, `hudi-kafka-connect-bundle` and
+ `hudi-integ-test-bundle`. Their `<filters>` currently exclude only signature files, `META-INF/services/javax.*` and
+ `**/*.proto`, so *any* resource under `src/main/resources/` is unpacked into each shaded JAR. The module has no
+ `src/main/resources` today, which is exactly why this has never bitten us - the ~890KB of vendored assets would be
+ net-new payload in all six bundles, not just the server bundle.
+
+ The first cut is standalone-only, so the UI cannot even be served from an embedded timeline server inside a Spark or
+ Flink driver. Shipping ~890KB of unreachable JS/CSS in five engine bundles is pure bloat, and it would drag the
+ LICENSE obligations below into each of them. The five non-server bundle poms therefore add an explicit shade filter:
+
+ ```xml
+ <filter>
+ <artifact>org.apache.hudi:hudi-timeline-service</artifact>
+ <excludes>
+ <exclude>public/**</exclude>
+ </excludes>
+ </filter>
+ ```
+
+ Only `hudi-timeline-server-bundle` ships the assets. When the embedded/Spark-UI follow-up lands, it must lift this
+ exclusion for the bundles that actually gain an embedded UI (`hudi-spark-bundle`, and `hudi-flink-bundle` if it
+ follows), and add the LICENSE stanzas below to those bundles at the same time. That is a deliberate, reviewable step
+ in the follow-up, not something to inherit silently here.
+- **LICENSE/NOTICE obligations.** Each vendored library needs a "This product bundles ..." stanza in the source-release
+ top-level `LICENSE` **and** in the LICENSE of every bundle whose JAR actually contains the assets, naming the library,
+ its license, and its copyright, with the full MIT/ISC license text inlined the same way existing bundled code is
+ handled in `LICENSE`. With the shade filter above, that set is exactly the source release plus
+ `hudi-timeline-server-bundle`; without it, it would be all six bundles. The minified files already carry their
+ upstream copyright headers, which must be preserved. No `NOTICE` changes are required: MIT and ISC do not mandate
+ NOTICE entries, and vis-timeline is taken under its MIT option (ASF policy discourages adding MIT/ISC copyrights to
+ `NOTICE`); were vis-timeline instead taken under Apache-2.0, any upstream `NOTICE` content it ships would have to be
+ propagated.
+- **Spark UI tab (planned follow-up):** Will use existing `spark-core` APIs (`WebUITab`, `WebUIPage`), already provided
+ in `hudi-spark-client`. No new JARs are added.
+- **No frontend build pipeline.** No npm, webpack, or vite. The JS/CSS files are committed directly and served as-is.
+
+## Additional Quality of Life (QoL) Features to be Considered
+
+The following client-side enhancements were implemented alongside the core features above. These are purely frontend
+additions - no new APIs, Java dependencies, or architectural changes beyond what is specified in this RFC.
+
+- **Bootstrap 5 + renderjson styling** - Bootstrap 5 for layout and renderjson for collapsible, syntax-highlighted JSON
+ in the detail panel.
+- **Instant search and "Go to Now" button** - A search input that filters and focuses the timeline on a specific instant
+ by requested time, plus a button to jump to the latest instant.
+- **Summary statistics bar** - Counts of instants by state (completed, inflight, requested) computed client-side from
+ the loaded data.
+- **Filter controls (state + action)** - Dropdown filters for state and action type using vis-timeline's `DataView`
+ filtering. Users can isolate specific instant types.
+- **Color legend** - Maps colors to instant states for quick reference.
+- **URL `?path=` persistence** - The basepath is stored in the URL query parameter so the view can be bookmarked and
+ shared.
+- **Keyboard navigation** - Left/right arrow keys to move between instants, Escape to close the detail panel.
+- **Help modal** - Pressing `?` opens a modal listing all keyboard shortcuts and UI interactions.
+
+## Future Improvements
+
+Other features we can add later:
+
+1. **Incremental clean range overlay** - A translucent overlay from `lastCompletedCommitTimestamp` to
+ `earliestCommitToRetain` when clean metadata is available, showing which instants are protected from cleaning and
+ which are eligible for removal.
+
+2. **Partition/filegroup write heatmap** - Within one or multiple instants, render a grid colored by write volume from
+ `HoodieCommitMetadata.partitionToWriteStats`. Helps identify skewed partitions or heavily-modified filegroups across
+ commits.
+
+3. **Metadata table timeline overlay** - Render the metadata table's timeline alongside the main table timeline. Useful
+ for diagnosing cases where the metadata table is lagging or has stale entries.
+
+4. **Archived timeline** - Load and render instants from the LSM-based archive files, extending the view beyond the
+ active timeline to include historical actions.
+
+5. **Per-state instant details** - Currently the plan for the UI is to only render the highest state for each instant
+ (e.g., if requested, inflight, and completed all exist, only completed is shown). We should keep this as the default
+ but also let users view the other states' metadata. The simplest approach would be adding state toggle buttons after
+ selecting an instant, so users can fetch and view the JSON content for each state individually.
+
+## Rollout/Adoption Plan
+
+- **Backward compatibility:** Additive change only. All existing `/v1/` endpoints remain unchanged. UI endpoints are
+ under `/v2/hoodie/view/` and the UI entry page at `/ui`. No existing behavior changes.
+- **Release target:** 1.2.x
+- **Documentation:** A new page on hudi.apache.org under the "Operations" section with screenshots, a startup guide, and
+ a debugging walkthrough showing how to use the timeline UI to diagnose common issues (stuck compactions, long-running
+ commits, etc.).
+- **No migration needed:** Net-new feature behind an opt-in flag (`--enable-ui`), no impact on existing users or running
+ systems.
+
+## Test Plan
+
+### Unit Tests
+
+- **`TimelineHandler.getTimelineV2()`**: Verify correct mapping from `HoodieInstant` to `InstantDTOV2`, including:
+ - Every action the active timeline can actually surface is mapped correctly. Note this is *not* one instant per
+ entry in `VALID_ACTIONS_IN_TIMELINE`: the layout filter collapses `(requestedTime, comparableAction)` groups, so
+ a completed `compaction`/`logcompaction`/`clustering` can only ever arrive as `commit`/`deltacommit`/
+ `replacecommit`, and a test that expects a completed instant with action `compaction` will never pass.
+ - `comparableAction` folds the pending actions correctly: a *pending* compaction maps to
+ (`action=compaction`, `comparableAction=commit`); pending logcompaction to `deltacommit`; pending clustering to
+ `replacecommit`. For every other action, `comparableAction` equals `action`.
+ - A compaction that has completed surfaces once, as `action=commit`, `state=COMPLETED` - not as three instants and
+ not with action `compaction`. Assert this against a table with a real completed compaction, since it is the case
+ the group model depends on.
+ - `completionTime` is populated for completed instants and null for requested/inflight instants.
+ - Instants are returned in timeline order.
+- **`TimelineHandler.getInstantDetails()`**: Verify correct Avro deserialization to JSON for various action types
+ (commit metadata, compaction plan, clean plan, etc.).
+- **`getInstantDetails()` path traversal (negative case):** an `instant` param of `../../../../tmp/x` (and similar
Review Comment:
This case only exercises the hole when `instantstate` is `REQUESTED` or `INFLIGHT`. `ActiveTimelineV2.getInstantFileName` already resolves a `COMPLETED` instant against the timeline when its completion time is null - which is exactly what a `HoodieInstant` reconstructed from request params looks like - and throws `HoodieIOException("Cannot find the instant ...")` rather than reaching `StoragePath`. A traversal probe sent with `instantstate=COMPLETED` therefore never touches the vulnerable path, and would pass against the unfixed handler.
Pin the negative case to the pending states, since those are the ones where the reconstructed instant goes straight to the file-name generator.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reads the table config / active timeline fresh on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server binds a free port by default
+(`hoodie.embed.timeline.server.port` = `0`), and the bound port is not written back to that property - read it from the
+startup log line (`Started embedded timeline server at <host>:<port>`), or from the resolved
+`FileSystemViewStorageConfig` remote-server port that `EmbeddedTimelineService` propagates it to.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, and each UI request builds a `HoodieTableMetaClient` for its
+own basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **The assets must be explicitly excluded from the engine bundles.** This does *not* come for free.
+ `hudi-timeline-service` is a compile-scope dependency of `hudi-client-common`, and six bundles list
+ `org.apache.hudi:hudi-timeline-service` in their shade `artifactSet`: `hudi-timeline-server-bundle` (intended) plus
+ `hudi-spark-bundle`, `hudi-flink-bundle`, `hudi-utilities-bundle`, `hudi-kafka-connect-bundle` and
+ `hudi-integ-test-bundle`. Their `<filters>` currently exclude only signature files, `META-INF/services/javax.*` and
+ `**/*.proto`, so *any* resource under `src/main/resources/` is unpacked into each shaded JAR. The module has no
+ `src/main/resources` today, which is exactly why this has never bitten us - the ~890KB of vendored assets would be
+ net-new payload in all six bundles, not just the server bundle.
+
+ The first cut is standalone-only, so the UI cannot even be served from an embedded timeline server inside a Spark or
+ Flink driver. Shipping ~890KB of unreachable JS/CSS in five engine bundles is pure bloat, and it would drag the
+ LICENSE obligations below into each of them. The five non-server bundle poms therefore add an explicit shade filter:
+
+ ```xml
+ <filter>
+ <artifact>org.apache.hudi:hudi-timeline-service</artifact>
+ <excludes>
+ <exclude>public/**</exclude>
+ </excludes>
+ </filter>
+ ```
+
+ Only `hudi-timeline-server-bundle` ships the assets. When the embedded/Spark-UI follow-up lands, it must lift this
+ exclusion for the bundles that actually gain an embedded UI (`hudi-spark-bundle`, and `hudi-flink-bundle` if it
+ follows), and add the LICENSE stanzas below to those bundles at the same time. That is a deliberate, reviewable step
+ in the follow-up, not something to inherit silently here.
+- **LICENSE/NOTICE obligations.** Each vendored library needs a "This product bundles ..." stanza in the source-release
+ top-level `LICENSE` **and** in the LICENSE of every bundle whose JAR actually contains the assets, naming the library,
+ its license, and its copyright, with the full MIT/ISC license text inlined the same way existing bundled code is
+ handled in `LICENSE`. With the shade filter above, that set is exactly the source release plus
+ `hudi-timeline-server-bundle`; without it, it would be all six bundles. The minified files already carry their
+ upstream copyright headers, which must be preserved. No `NOTICE` changes are required: MIT and ISC do not mandate
+ NOTICE entries, and vis-timeline is taken under its MIT option (ASF policy discourages adding MIT/ISC copyrights to
+ `NOTICE`); were vis-timeline instead taken under Apache-2.0, any upstream `NOTICE` content it ships would have to be
+ propagated.
+- **Spark UI tab (planned follow-up):** Will use existing `spark-core` APIs (`WebUITab`, `WebUIPage`), already provided
+ in `hudi-spark-client`. No new JARs are added.
+- **No frontend build pipeline.** No npm, webpack, or vite. The JS/CSS files are committed directly and served as-is.
+
+## Additional Quality of Life (QoL) Features to be Considered
+
+The following client-side enhancements were implemented alongside the core features above. These are purely frontend
+additions - no new APIs, Java dependencies, or architectural changes beyond what is specified in this RFC.
+
+- **Bootstrap 5 + renderjson styling** - Bootstrap 5 for layout and renderjson for collapsible, syntax-highlighted JSON
+ in the detail panel.
+- **Instant search and "Go to Now" button** - A search input that filters and focuses the timeline on a specific instant
+ by requested time, plus a button to jump to the latest instant.
+- **Summary statistics bar** - Counts of instants by state (completed, inflight, requested) computed client-side from
+ the loaded data.
+- **Filter controls (state + action)** - Dropdown filters for state and action type using vis-timeline's `DataView`
+ filtering. Users can isolate specific instant types.
+- **Color legend** - Maps colors to instant states for quick reference.
+- **URL `?path=` persistence** - The basepath is stored in the URL query parameter so the view can be bookmarked and
+ shared.
+- **Keyboard navigation** - Left/right arrow keys to move between instants, Escape to close the detail panel.
+- **Help modal** - Pressing `?` opens a modal listing all keyboard shortcuts and UI interactions.
+
+## Future Improvements
+
+Other features we can add later:
+
+1. **Incremental clean range overlay** - A translucent overlay from `lastCompletedCommitTimestamp` to
+ `earliestCommitToRetain` when clean metadata is available, showing which instants are protected from cleaning and
+ which are eligible for removal.
+
+2. **Partition/filegroup write heatmap** - Within one or multiple instants, render a grid colored by write volume from
+ `HoodieCommitMetadata.partitionToWriteStats`. Helps identify skewed partitions or heavily-modified filegroups across
+ commits.
+
+3. **Metadata table timeline overlay** - Render the metadata table's timeline alongside the main table timeline. Useful
+ for diagnosing cases where the metadata table is lagging or has stale entries.
+
+4. **Archived timeline** - Load and render instants from the LSM-based archive files, extending the view beyond the
+ active timeline to include historical actions.
+
+5. **Per-state instant details** - Currently the plan for the UI is to only render the highest state for each instant
+ (e.g., if requested, inflight, and completed all exist, only completed is shown). We should keep this as the default
+ but also let users view the other states' metadata. The simplest approach would be adding state toggle buttons after
+ selecting an instant, so users can fetch and view the JSON content for each state individually.
+
+## Rollout/Adoption Plan
+
+- **Backward compatibility:** Additive change only. All existing `/v1/` endpoints remain unchanged. UI endpoints are
+ under `/v2/hoodie/view/` and the UI entry page at `/ui`. No existing behavior changes.
+- **Release target:** 1.2.x
+- **Documentation:** A new page on hudi.apache.org under the "Operations" section with screenshots, a startup guide, and
+ a debugging walkthrough showing how to use the timeline UI to diagnose common issues (stuck compactions, long-running
+ commits, etc.).
+- **No migration needed:** Net-new feature behind an opt-in flag (`--enable-ui`), no impact on existing users or running
+ systems.
+
+## Test Plan
+
+### Unit Tests
+
+- **`TimelineHandler.getTimelineV2()`**: Verify correct mapping from `HoodieInstant` to `InstantDTOV2`, including:
+ - Every action the active timeline can actually surface is mapped correctly. Note this is *not* one instant per
+ entry in `VALID_ACTIONS_IN_TIMELINE`: the layout filter collapses `(requestedTime, comparableAction)` groups, so
+ a completed `compaction`/`logcompaction`/`clustering` can only ever arrive as `commit`/`deltacommit`/
+ `replacecommit`, and a test that expects a completed instant with action `compaction` will never pass.
+ - `comparableAction` folds the pending actions correctly: a *pending* compaction maps to
+ (`action=compaction`, `comparableAction=commit`); pending logcompaction to `deltacommit`; pending clustering to
+ `replacecommit`. For every other action, `comparableAction` equals `action`.
+ - A compaction that has completed surfaces once, as `action=commit`, `state=COMPLETED` - not as three instants and
+ not with action `compaction`. Assert this against a table with a real completed compaction, since it is the case
+ the group model depends on.
+ - `completionTime` is populated for completed instants and null for requested/inflight instants.
+ - Instants are returned in timeline order.
+- **`TimelineHandler.getInstantDetails()`**: Verify correct Avro deserialization to JSON for various action types
+ (commit metadata, compaction plan, clean plan, etc.).
+- **`getInstantDetails()` path traversal (negative case):** an `instant` param of `../../../../tmp/x` (and similar
+ `..`-bearing values) must return 404 and must not read outside the table's timeline directory. Plant a readable file
+ at the traversal target and assert its content never appears in the response. A well-formed timestamp that is not in
+ the active timeline must also return 404. This is the assertion the instant-detail route's safety rests on: it can
+ only read instants the timeline actually lists.
+- **Error cases**: Invalid basepath returns an appropriate error. Empty timeline returns an empty list.
+
+### Integration Tests
+
+Following the pattern established by `TestTimelineService.java`:
+
+- Start `TimelineService` with the UI enabled (`--enable-ui`).
+- Create synthetic instants via `HoodieTestUtils` covering multiple action types and states.
+- Verify `GET /v2/hoodie/view/timeline/instants/all` returns the expected list of instants with correct fields.
+- Verify `GET /v2/hoodie/view/timeline/instant` returns valid JSON for a completed instant.
+- Verify `GET /v2/hoodie/view/table/config` and `GET /v2/hoodie/view/table/schema/history` return valid JSON.
+- Verify `GET /ui` returns HTTP 200 with HTML content (static file serving works).
+- **Flag gating (negative case):** start `TimelineService` *without* `--enable-ui` (the default) and assert `GET /ui`,
+ `GET /ui/static/*` and every `GET /v2/hoodie/view/*` route return HTTP 404, while the existing `/v1/` routes still
+ respond. This is the assertion the Scope section's threat model rests on - the widened read surface must not exist
+ unless the flag is explicitly set.
+
+### Bundle Packaging
+
+The `public/**` shade exclusion in [Dependency Impact](#dependency-impact) is silent when it regresses: drop the filter
+and the assets simply reappear in every engine bundle, with nothing failing. Guard it in `packaging/bundle-validation`:
Review Comment:
`packaging/bundle-validation` cannot host either assertion as written. `ci_run.sh` copies only the flink, kafka-connect, metaserver-server, cli, hadoop-mr, spark, utilities and utilities-slim bundles into the validation image - neither `hudi-timeline-server-bundle` nor `hudi-integ-test-bundle` is wired in, so the positive assertion (the one proving the assets ship at all) and one of the negative ones have no bundle to run against.
The harness also has no jar-content assertion primitive: `validate.sh` is a Docker runtime smoke test (`test_spark_hadoop_mr_bundles`, `test_utilities_bundle`, `test_flink_bundle`, `test_kafka_connect_bundle`, ...), and there is no `jar tf` or `unzip -l` anywhere under `packaging/bundle-validation`.
So this guard is net-new tooling plus two bundles newly wired into CI, not a small addition to an existing pattern. Either say that, or put the check somewhere jar contents are already inspectable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315][RFC-94] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3401323530
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
+The vis-timeline library is loaded with a two-tier strategy:
+
+1. **Primary:** Load from the `unpkg.com` CDN for automatic patch updates.
+2. **Fallback:** If the CDN is unreachable (e.g., air-gapped environments), load from the bundled
+ copy at `/lib/vis-timeline/`.
+
+The bundled fallback adds ~300KB to the JAR but ensures the UI works without internet access.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`,
+ `clustering`, `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These
+ correspond to the actions in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to
+ `completionTime`. Non-completed instants (requested or inflight) are rendered as point items at
+ `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/timeline/instant/details` and
+ shows the deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths
+since the `FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|--------------------------------|----------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------------------|
+| GET | `/v2/timeline/instants` | `basepath` (required) | `List<InstantDTO>` (v2) | Returns all active instants with requested time, completion time, action, and state |
+| GET | `/v2/timeline/instant/details` | `basepath` (required), `instantTime` (required), `action` (required) | JSON string | Returns the deserialized content of a specific instant's metadata (Avro -> JSON) |
+
Review Comment:
Good point. Scoping note: this endpoint serves the active timeline only (the unbounded archived timeline is out of scope), and the active timeline is bounded by archiving, so counts are usually modest. The first cut returns all active instants and uses client-side zoom/scroll + filtering for navigation. Added a note to the API spec documenting that intent and that the endpoint can be extended additively with optional `from`/`to` time-range params (and/or a `limit`) if active-timeline sizes ever become a concern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3563380805
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
Review Comment:
Good catch -- fixed. Bootstrap 5 and renderjson are both vendored under `lib/` alongside vis-timeline. Updated the File Structure tree and Dependency Impact to list all three:
| Library | Size | License |
|---|---|---|
| vis-timeline | ~575KB | Apache-2.0 OR MIT |
| Bootstrap 5 | ~305KB | MIT |
| renderjson | ~11KB | ISC |
Also corrected the footprint -- the ~300KB was vis-timeline-only; actual total is ~890KB. Added a LICENSE/NOTICE bullet: all three are Category A, so each needs a `This product bundles ...` stanza in the top-level `LICENSE` and the `hudi-timeline-server-bundle` LICENSE. No `NOTICE` changes needed -- MIT/ISC don't require them and vis-timeline is taken under its MIT option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3563467309
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
Review Comment:
Good callout, in my current implementation, we are indeed serving hoodie.properties.
Maybe we should add a note here to not serve the full list of `hoodie.properties` if we want to render it. The current deployment model is for users that are already able to view `hoodie.properties` un-redacted. So, reserving them should be a non-issue.
Nonetheless, i added a note in the RFC for future improvements to add in untrusted or less secure modes of serving.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4943210681
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14643",
"triggerID" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"triggerType" : "PUSH"
}, {
"hash" : "4e021439627242ed5605c1b376e662ba405c0573",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14646",
"triggerID" : "4e021439627242ed5605c1b376e662ba405c0573",
"triggerType" : "PUSH"
}, {
"hash" : "3b093ed70674e1057c1ffe828018f762aeb9e8ba",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3b093ed70674e1057c1ffe828018f762aeb9e8ba",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4e021439627242ed5605c1b376e662ba405c0573 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14646)
* 3b093ed70674e1057c1ffe828018f762aeb9e8ba UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3569305168
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
Review Comment:
You are right, and the sentence was uncallable as written -- `InstantComparator` declares only `actionOnlyComparator`/`requestedTimeOrderedComparator`/`completionTimeOrderedComparator`, and a `public static` on the impls is not reachable through an interface-typed reference. Confirmed there is no version-dispatched accessor either: `TimelineLayout` exposes `getInstantComparator()` but not the mapper, `filterHoodieInstantsByLatestState` is `private`, and `metaClient.timelineLayout` is a private field with no getter.
Took your first option -- pinned `InstantComparatorV2.getComparableAction(action)` -- and wrote the safety argument into the RFC rather than leaving it implicit, since it is subtle. Verified both halves of it:
- `InstantComparatorV1`'s map has only `compaction -> commit` and `logcompaction -> deltacommit`; no `clustering` entry.
- `ActiveTimelineV1.VALID_EXTENSIONS_IN_ACTIVE_TIMELINE` (`:67-78`) contains no clustering extension.
So a layout-v1 timeline can never surface an instant the two maps disagree on.
Deliberately **not** taking the second option here. Lifting `getComparableAction` onto `InstantComparator`/`TimelineLayout` is the right cleanup, but it changes a shared `hudi-common` interface to serve a UI display field, and this RFC has exactly one caller. Noted it in the RFC as the follow-up if a second caller appears -- happy to be overruled if you feel the API belongs there regardless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3563455915
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
Review Comment:
Done -- namespaced the static assets under `/ui/static/` (Javalin `hostedPath`), so the whole UI surface sits under `/ui` and the root `/js`, `/css`, `/lib` prefixes stay free for `/v1/`, `/v2/`, and any future routes. Assets now serve at `/ui/static/js/...`, `/ui/static/lib/...`, etc., with `index.html` links repointed accordingly; API routes under `/v1/` and `/v2/` are unchanged. Updated the architecture diagram, request-flow description, JavaScript-delivery paths, and the API-spec static-file note.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3565977223
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+Both `getTableConfig` and `getSchemaHistory` reuse a `HoodieTableMetaClient` cached per basepath in a
+`ConcurrentHashMap`, so repeated requests skip full metaClient construction. A `HoodieTableMetaClient` snapshots its
+`tableConfig` and `activeTimeline` at construction and refreshes them only on demand, so a build-once cache would serve
+stale config/schema for the lifetime of a long-lived embedded driver. To avoid that, each config/schema request first
+reloads the relevant snapshot under a per-basepath guard - `reloadTableConfig()` for `getTableConfig`,
+`reloadActiveTimeline()` for `getSchemaHistory` - before reading. These are human-click-frequency views, so the reload
+cost (one `hoodie.properties` read plus one timeline listing) is negligible and the response is always current. A TTL is
+deliberately not used: at this request rate it would save no meaningful work and would only reintroduce a staleness
+window. The UI also surfaces a **Refresh** control on the Table Config and Schema History tabs to re-pull on demand;
+because the server reloads on every request, the button simply re-issues the fetch rather than busting a cache. The
+timeline and instant-detail views are unaffected - they read the server-synced `FileSystemView`, not the cached
+metaClient.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reloads the table config / active timeline on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server port is logged at startup and can also be
+retrieved from the Spark configuration via the `hoodie.embed.timeline.server.port` property.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **LICENSE/NOTICE obligations.** Each vendored library needs a "This product bundles ..." stanza in the source-release
+ top-level `LICENSE` (and in the `hudi-timeline-server-bundle` LICENSE, since the assets ship inside that JAR) naming
+ the library, its license, and its copyright, with the full MIT/ISC license text inlined the same way existing bundled
+ code is handled in `LICENSE`. The minified files already carry their upstream copyright headers, which must be
+ preserved. No `NOTICE` changes are required: MIT and ISC do not mandate NOTICE entries, and vis-timeline is taken
+ under its MIT option (ASF policy discourages adding MIT/ISC copyrights to `NOTICE`); were vis-timeline instead taken
+ under Apache-2.0, any upstream `NOTICE` content it ships would have to be propagated.
+- **Spark UI tab (planned follow-up):** Will use existing `spark-core` APIs (`WebUITab`, `WebUIPage`), already provided
+ in `hudi-spark-client`. No new JARs are added.
+- **No impact on Spark/Flink bundles.** `hudi-timeline-server-bundle` is a separate artifact; adding static resources
+ does not affect engine-specific bundles.
+- **No frontend build pipeline.** No npm, webpack, or vite. The JS/CSS files are committed directly and served as-is.
+
+## Additional Quality of Life (QoL) Features to be Considered
+
+The following client-side enhancements were implemented alongside the core features above. These are purely frontend
+additions - no new APIs, Java dependencies, or architectural changes beyond what is specified in this RFC.
+
+- **Bootstrap 5 + renderjson styling** - Bootstrap 5 for layout and renderjson for collapsible, syntax-highlighted JSON
+ in the detail panel.
+- **Instant search and "Go to Now" button** - A search input that filters and focuses the timeline on a specific instant
+ by requested time, plus a button to jump to the latest instant.
+- **Summary statistics bar** - Counts of instants by state (completed, inflight, requested) computed client-side from
+ the loaded data.
+- **Filter controls (state + action)** - Dropdown filters for state and action type using vis-timeline's `DataView`
+ filtering. Users can isolate specific instant types.
+- **Color legend** - Maps colors to instant states for quick reference.
+- **URL `?path=` persistence** - The basepath is stored in the URL query parameter so the view can be bookmarked and
+ shared.
+- **Keyboard navigation** - Left/right arrow keys to move between instants, Escape to close the detail panel.
+- **Help modal** - Pressing `?` opens a modal listing all keyboard shortcuts and UI interactions.
+
+## Future Improvements
+
+Other features we can add later:
+
+1. **Incremental clean range overlay** - A translucent overlay from `lastCompletedCommitTimestamp` to
+ `earliestCommitToRetain` when clean metadata is available, showing which instants are protected from cleaning and
+ which are eligible for removal.
+
+2. **Partition/filegroup write heatmap** - Within one or multiple instants, render a grid colored by write volume from
+ `HoodieCommitMetadata.partitionToWriteStats`. Helps identify skewed partitions or heavily-modified filegroups across
+ commits.
+
+3. **Metadata table timeline overlay** - Render the metadata table's timeline alongside the main table timeline. Useful
+ for diagnosing cases where the metadata table is lagging or has stale entries.
+
+4. **Archived timeline** - Load and render instants from the LSM-based archive files, extending the view beyond the
+ active timeline to include historical actions.
+
+5. **Per-state instant details** - Currently the plan for the UI is to only render the highest state for each instant
+ (e.g., if requested, inflight, and completed all exist, only completed is shown). We should keep this as the default
+ but also let users view the other states' metadata. The simplest approach would be adding state toggle buttons after
+ selecting an instant, so users can fetch and view the JSON content for each state individually.
+
+## Rollout/Adoption Plan
+
+- **Backward compatibility:** Additive change only. All existing `/v1/` endpoints remain unchanged. UI endpoints are
+ under `/v2/hoodie/view/` and the UI entry page at `/ui`. No existing behavior changes.
+- **Release target:** 1.2.x
+- **Documentation:** A new page on hudi.apache.org under the "Operations" section with screenshots, a startup guide, and
+ a debugging walkthrough showing how to use the timeline UI to diagnose common issues (stuck compactions, long-running
+ commits, etc.).
+- **No migration needed:** Net-new feature behind an opt-in flag (`--enable-ui`), no impact on existing users or running
+ systems.
+
+## Test Plan
+
+### Unit Tests
+
+- **`TimelineHandler.getTimelineV2()`**: Verify correct mapping from `HoodieInstant` to v2 `InstantDTO`, including:
+ - All action types in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` are mapped correctly.
+ - `completionTime` is populated for completed instants and null for requested/inflight instants.
+ - Instants are returned in timeline order.
+- **`TimelineHandler.getInstantDetails()`**: Verify correct Avro deserialization to JSON for various action types
+ (commit metadata, compaction plan, clean plan, etc.).
+- **Error cases**: Invalid basepath returns an appropriate error. Empty timeline returns an empty list.
+
+### Integration Tests
+
+Following the pattern established by `TestTimelineService.java`:
+
+- Start `TimelineService` with the UI enabled (`--enable-ui`).
Review Comment:
Good point -- the security argument was untested. Added the negative integration case: start `TimelineService` *without* `--enable-ui` (the default) and assert `GET /ui`, `GET /ui/static/*` and every `GET /v2/hoodie/view/*` route return 404, while the `/v1/` routes still respond. That last clause pins that only the widened UI surface is gated, not the whole server.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3564128089
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
Review Comment:
This PR lands the doc but leaves the `rfc/README.md` index row for 94 as the un-linked reservation stub (`| 94 | Hudi Timeline User Interface (UI) | :eyes: UNDER REVIEW |`). `rfc/template.md` asks contributors to keep the status updated in `rfc/README.md`, and the doc-landing PRs for peer RFCs flip that row to a link in the same commit (RFC-101 in c02706ed, RFC-93 in 720e7eb9).
Add to this PR: `| 94 | [Hudi Timeline User Interface (UI)](./rfc-94/rfc-94.md) | :eyes: UNDER REVIEW |`.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
Review Comment:
Within one new `dto.v2` package, `InstantDTO` is suffix-free while its sibling is `TimelineDTOV2`. The repo's settled convention for versioned timeline classes is package *and* suffix uniformly: `versioning/v1/` and `versioning/v2/` hold `InstantGeneratorV1`/`InstantGeneratorV2`, `BaseTimelineV1`/`BaseTimelineV2`, `CommitMetadataSerDeV1`/`CommitMetadataSerDeV2`.
The suffix-free name also collides on simple name with the existing `dto.InstantDTO`, which `TimelineHandler` already imports (TimelineHandler.java:21) and will keep importing since it serves both v1 and v2 routes, forcing a fully-qualified name in that class. Pick one scheme: `dto.v2.{InstantDTOV2, TimelineDTOV2}` (matches the repo) or `dto.v2.{InstantDTO, TimelineDTO}` (pure package namespacing).
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
Review Comment:
`getFileSystemView(basePath).getTimeline()` cannot serve the timeline this RFC renders. `AbstractTableFileSystemView.getTimeline()` returns `visibleCommitsAndCompactionTimeline`, which `refreshTimeline()` sets to `visibleActiveTimeline.getWriteTimeline()`, and `getWriteTimeline()` keeps only `commit`, `deltacommit`, `compaction`, `logcompaction`, `replacecommit`, `clustering` (BaseHoodieTimeline.java:200-202). The view's `visibleActiveTimeline` is itself pre-filtered to `isCompleted() || action == compaction` (FileSystemViewManager.java:174, and `filterCompletedOrMajorOrMinorCompactionInstants()` on every `sync()`).
That source can never return `clean`, `rollback`, `savepoint`, `restore` or `indexing` in any state, nor requested/inflight `commit`/`deltacommit`/`replacecommit`/`clustering`. It contradicts the vis-timeline spec above (one group per action in `VALID_ACTIONS_IN_TIMELINE`, requested/inflight as point items, yellow/red), the Test Plan (`completionTime` null for requested/inflight instants), and the manual checklist (clean and rollback must render). Only compaction and logcompaction keep their pending states, which is likely why the PoC screenshots look correct.
Read the active timeline directly instead: the per-basepath `HoodieTableMetaClient` this section already introduces for config/schema exposes `getActiveTimeline()`, which carries all 11 actions in all states.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
Review Comment:
`getInstantDetails()` is `@Deprecated` on `HoodieInstantReader` (HoodieInstantReader.java:47-48); the replacement is `getContentStream(HoodieInstant)` in the same file (line 38). A net-new `/v2/` endpoint should not be specified against the deprecated method.
Separately, "the timeline's own layout-aware `InstantGenerator`" implies an accessor that does not exist: the field is `protected` on `BaseHoodieTimeline`, and the public sources are `HoodieTableMetaClient.getInstantGenerator()` or `TimelineLayout.fromVersion(...).getInstantGenerator()`. Worth naming the one you intend, since the handler already holds the metaClient.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+Both `getTableConfig` and `getSchemaHistory` reuse a `HoodieTableMetaClient` cached per basepath in a
+`ConcurrentHashMap`, so repeated requests skip full metaClient construction. A `HoodieTableMetaClient` snapshots its
+`tableConfig` and `activeTimeline` at construction and refreshes them only on demand, so a build-once cache would serve
+stale config/schema for the lifetime of a long-lived embedded driver. To avoid that, each config/schema request first
+reloads the relevant snapshot under a per-basepath guard - `reloadTableConfig()` for `getTableConfig`,
+`reloadActiveTimeline()` for `getSchemaHistory` - before reading. These are human-click-frequency views, so the reload
+cost (one `hoodie.properties` read plus one timeline listing) is negligible and the response is always current. A TTL is
+deliberately not used: at this request rate it would save no meaningful work and would only reintroduce a staleness
+window. The UI also surfaces a **Refresh** control on the Table Config and Schema History tabs to re-pull on demand;
+because the server reloads on every request, the button simply re-issues the fetch rather than busting a cache. The
+timeline and instant-detail views are unaffected - they read the server-synced `FileSystemView`, not the cached
+metaClient.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reloads the table config / active timeline on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server port is logged at startup and can also be
+retrieved from the Spark configuration via the `hoodie.embed.timeline.server.port` property.
Review Comment:
`hoodie.embed.timeline.server.port` defaults to `0` ("picks a free port", HoodieWriteConfig.java:595-599), and the bound port is never written back to it: `EmbeddedTimelineService` assigns `serverPort = server.startService()` (EmbeddedTimelineService.java:180), logs it (line 181), and propagates it only through `FileSystemViewStorageConfig.withRemoteServerPort()` (line 210). A user who follows this sentence and reads the property back gets `0`, not the live port.
Point users at the startup log line or at the view-storage config, and drop the "retrieved from the Spark configuration" claim.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+Both `getTableConfig` and `getSchemaHistory` reuse a `HoodieTableMetaClient` cached per basepath in a
+`ConcurrentHashMap`, so repeated requests skip full metaClient construction. A `HoodieTableMetaClient` snapshots its
+`tableConfig` and `activeTimeline` at construction and refreshes them only on demand, so a build-once cache would serve
+stale config/schema for the lifetime of a long-lived embedded driver. To avoid that, each config/schema request first
+reloads the relevant snapshot under a per-basepath guard - `reloadTableConfig()` for `getTableConfig`,
+`reloadActiveTimeline()` for `getSchemaHistory` - before reading. These are human-click-frequency views, so the reload
+cost (one `hoodie.properties` read plus one timeline listing) is negligible and the response is always current. A TTL is
+deliberately not used: at this request rate it would save no meaningful work and would only reintroduce a staleness
+window. The UI also surfaces a **Refresh** control on the Table Config and Schema History tabs to re-pull on demand;
+because the server reloads on every request, the button simply re-issues the fetch rather than busting a cache. The
+timeline and instant-detail views are unaffected - they read the server-synced `FileSystemView`, not the cached
+metaClient.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reloads the table config / active timeline on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server port is logged at startup and can also be
+retrieved from the Spark configuration via the `hoodie.embed.timeline.server.port` property.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **LICENSE/NOTICE obligations.** Each vendored library needs a "This product bundles ..." stanza in the source-release
+ top-level `LICENSE` (and in the `hudi-timeline-server-bundle` LICENSE, since the assets ship inside that JAR) naming
+ the library, its license, and its copyright, with the full MIT/ISC license text inlined the same way existing bundled
+ code is handled in `LICENSE`. The minified files already carry their upstream copyright headers, which must be
+ preserved. No `NOTICE` changes are required: MIT and ISC do not mandate NOTICE entries, and vis-timeline is taken
+ under its MIT option (ASF policy discourages adding MIT/ISC copyrights to `NOTICE`); were vis-timeline instead taken
+ under Apache-2.0, any upstream `NOTICE` content it ships would have to be propagated.
+- **Spark UI tab (planned follow-up):** Will use existing `spark-core` APIs (`WebUITab`, `WebUIPage`), already provided
+ in `hudi-spark-client`. No new JARs are added.
+- **No impact on Spark/Flink bundles.** `hudi-timeline-server-bundle` is a separate artifact; adding static resources
+ does not affect engine-specific bundles.
+- **No frontend build pipeline.** No npm, webpack, or vite. The JS/CSS files are committed directly and served as-is.
+
+## Additional Quality of Life (QoL) Features to be Considered
+
+The following client-side enhancements were implemented alongside the core features above. These are purely frontend
+additions - no new APIs, Java dependencies, or architectural changes beyond what is specified in this RFC.
+
+- **Bootstrap 5 + renderjson styling** - Bootstrap 5 for layout and renderjson for collapsible, syntax-highlighted JSON
+ in the detail panel.
+- **Instant search and "Go to Now" button** - A search input that filters and focuses the timeline on a specific instant
+ by requested time, plus a button to jump to the latest instant.
+- **Summary statistics bar** - Counts of instants by state (completed, inflight, requested) computed client-side from
+ the loaded data.
+- **Filter controls (state + action)** - Dropdown filters for state and action type using vis-timeline's `DataView`
+ filtering. Users can isolate specific instant types.
+- **Color legend** - Maps colors to instant states for quick reference.
+- **URL `?path=` persistence** - The basepath is stored in the URL query parameter so the view can be bookmarked and
+ shared.
+- **Keyboard navigation** - Left/right arrow keys to move between instants, Escape to close the detail panel.
+- **Help modal** - Pressing `?` opens a modal listing all keyboard shortcuts and UI interactions.
+
+## Future Improvements
+
+Other features we can add later:
+
+1. **Incremental clean range overlay** - A translucent overlay from `lastCompletedCommitTimestamp` to
+ `earliestCommitToRetain` when clean metadata is available, showing which instants are protected from cleaning and
+ which are eligible for removal.
+
+2. **Partition/filegroup write heatmap** - Within one or multiple instants, render a grid colored by write volume from
+ `HoodieCommitMetadata.partitionToWriteStats`. Helps identify skewed partitions or heavily-modified filegroups across
+ commits.
+
+3. **Metadata table timeline overlay** - Render the metadata table's timeline alongside the main table timeline. Useful
+ for diagnosing cases where the metadata table is lagging or has stale entries.
+
+4. **Archived timeline** - Load and render instants from the LSM-based archive files, extending the view beyond the
+ active timeline to include historical actions.
+
+5. **Per-state instant details** - Currently the plan for the UI is to only render the highest state for each instant
+ (e.g., if requested, inflight, and completed all exist, only completed is shown). We should keep this as the default
+ but also let users view the other states' metadata. The simplest approach would be adding state toggle buttons after
+ selecting an instant, so users can fetch and view the JSON content for each state individually.
+
+## Rollout/Adoption Plan
+
+- **Backward compatibility:** Additive change only. All existing `/v1/` endpoints remain unchanged. UI endpoints are
+ under `/v2/hoodie/view/` and the UI entry page at `/ui`. No existing behavior changes.
+- **Release target:** 1.2.x
+- **Documentation:** A new page on hudi.apache.org under the "Operations" section with screenshots, a startup guide, and
+ a debugging walkthrough showing how to use the timeline UI to diagnose common issues (stuck compactions, long-running
+ commits, etc.).
+- **No migration needed:** Net-new feature behind an opt-in flag (`--enable-ui`), no impact on existing users or running
+ systems.
+
+## Test Plan
+
+### Unit Tests
+
+- **`TimelineHandler.getTimelineV2()`**: Verify correct mapping from `HoodieInstant` to v2 `InstantDTO`, including:
+ - All action types in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` are mapped correctly.
+ - `completionTime` is populated for completed instants and null for requested/inflight instants.
+ - Instants are returned in timeline order.
+- **`TimelineHandler.getInstantDetails()`**: Verify correct Avro deserialization to JSON for various action types
+ (commit metadata, compaction plan, clean plan, etc.).
+- **Error cases**: Invalid basepath returns an appropriate error. Empty timeline returns an empty list.
+
+### Integration Tests
+
+Following the pattern established by `TestTimelineService.java`:
+
+- Start `TimelineService` with the UI enabled (`--enable-ui`).
Review Comment:
The threat model calls `--enable-ui` (off by default) the primary control for the two views that widen the read surface beyond `/v1/`, but every test here starts the service *with* the flag set. There is no negative case asserting that `/ui`, `/ui/static/*` and `/v2/hoodie/view/*` return 404 when the flag is unset.
Add that case: it is the single assertion the Scope section's security argument rests on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3565936248
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
Review Comment:
Good point. Went with `dto.v2.{InstantDTOV2, TimelineDTOV2}` -- matches the repo's versioned-class convention (`InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and avoids the simple-name clash with the existing `dto.InstantDTO` that `TimelineHandler` keeps importing for the v1 routes.
Renamed the v2 references in the DTO Design and Test Plan sections (the v1 `InstantDTO` mention stays as-is) and added a line documenting why both v2 DTOs carry the suffix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4956827059
I've reorganized the RFC such that implementation that we thought and considered are stored in the appendix section.
For pure design considerations and details, i've included it in the main section so that it's easier to follow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3569413150
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reads the table config / active timeline fresh on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server binds a free port by default
+(`hoodie.embed.timeline.server.port` = `0`), and the bound port is not written back to that property - read it from the
+startup log line (`Started embedded timeline server at <host>:<port>`), or from the resolved
+`FileSystemViewStorageConfig` remote-server port that `EmbeddedTimelineService` propagates it to.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, and each UI request builds a `HoodieTableMetaClient` for its
+own basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **The assets must be explicitly excluded from the engine bundles.** This does *not* come for free.
+ `hudi-timeline-service` is a compile-scope dependency of `hudi-client-common`, and six bundles list
+ `org.apache.hudi:hudi-timeline-service` in their shade `artifactSet`: `hudi-timeline-server-bundle` (intended) plus
+ `hudi-spark-bundle`, `hudi-flink-bundle`, `hudi-utilities-bundle`, `hudi-kafka-connect-bundle` and
+ `hudi-integ-test-bundle`. Their `<filters>` currently exclude only signature files, `META-INF/services/javax.*` and
+ `**/*.proto`, so *any* resource under `src/main/resources/` is unpacked into each shaded JAR. The module has no
+ `src/main/resources` today, which is exactly why this has never bitten us - the ~890KB of vendored assets would be
+ net-new payload in all six bundles, not just the server bundle.
+
+ The first cut is standalone-only, so the UI cannot even be served from an embedded timeline server inside a Spark or
+ Flink driver. Shipping ~890KB of unreachable JS/CSS in five engine bundles is pure bloat, and it would drag the
+ LICENSE obligations below into each of them. The five non-server bundle poms therefore add an explicit shade filter:
+
+ ```xml
+ <filter>
+ <artifact>org.apache.hudi:hudi-timeline-service</artifact>
+ <excludes>
+ <exclude>public/**</exclude>
+ </excludes>
+ </filter>
+ ```
+
+ Only `hudi-timeline-server-bundle` ships the assets. When the embedded/Spark-UI follow-up lands, it must lift this
Review Comment:
Confirmed, and this is a hazard my own shade filter introduced -- thanks for catching it. Verified the failure mode: `startServiceOnPort` catches `Exception`, and only `JavalinBindException` gets the "port busy" branch; everything else logs *"Timeline server start failed on port N. Attempting port N + 1"*, retries `START_SERVICE_MAX_RETRIES` (16), then throws `IOException("...after retry 16 times")`. Javalin 4.6.7 resolves a `Location.CLASSPATH` dir eagerly at `Javalin.create()`, i.e. inside `createApp()`. So a missing `/public` would surface as a bogus port failure.
Agreed the flag can no longer be the only gate once assets and code ship separately. RFC now specifies static-file registration conditional on `TimelineService.class.getResource("/public") != null`, failing fast with an explicit "UI assets not bundled in this jar" message, and notes the embedded follow-up sharpens this rather than removing it.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reads the table config / active timeline fresh on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server binds a free port by default
+(`hoodie.embed.timeline.server.port` = `0`), and the bound port is not written back to that property - read it from the
+startup log line (`Started embedded timeline server at <host>:<port>`), or from the resolved
+`FileSystemViewStorageConfig` remote-server port that `EmbeddedTimelineService` propagates it to.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, and each UI request builds a `HoodieTableMetaClient` for its
+own basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **The assets must be explicitly excluded from the engine bundles.** This does *not* come for free.
+ `hudi-timeline-service` is a compile-scope dependency of `hudi-client-common`, and six bundles list
+ `org.apache.hudi:hudi-timeline-service` in their shade `artifactSet`: `hudi-timeline-server-bundle` (intended) plus
+ `hudi-spark-bundle`, `hudi-flink-bundle`, `hudi-utilities-bundle`, `hudi-kafka-connect-bundle` and
+ `hudi-integ-test-bundle`. Their `<filters>` currently exclude only signature files, `META-INF/services/javax.*` and
+ `**/*.proto`, so *any* resource under `src/main/resources/` is unpacked into each shaded JAR. The module has no
+ `src/main/resources` today, which is exactly why this has never bitten us - the ~890KB of vendored assets would be
+ net-new payload in all six bundles, not just the server bundle.
+
+ The first cut is standalone-only, so the UI cannot even be served from an embedded timeline server inside a Spark or
+ Flink driver. Shipping ~890KB of unreachable JS/CSS in five engine bundles is pure bloat, and it would drag the
+ LICENSE obligations below into each of them. The five non-server bundle poms therefore add an explicit shade filter:
+
+ ```xml
+ <filter>
+ <artifact>org.apache.hudi:hudi-timeline-service</artifact>
+ <excludes>
+ <exclude>public/**</exclude>
+ </excludes>
+ </filter>
+ ```
+
+ Only `hudi-timeline-server-bundle` ships the assets. When the embedded/Spark-UI follow-up lands, it must lift this
+ exclusion for the bundles that actually gain an embedded UI (`hudi-spark-bundle`, and `hudi-flink-bundle` if it
+ follows), and add the LICENSE stanzas below to those bundles at the same time. That is a deliberate, reviewable step
+ in the follow-up, not something to inherit silently here.
+- **LICENSE/NOTICE obligations.** Each vendored library needs a "This product bundles ..." stanza in the source-release
+ top-level `LICENSE` **and** in the LICENSE of every bundle whose JAR actually contains the assets, naming the library,
+ its license, and its copyright, with the full MIT/ISC license text inlined the same way existing bundled code is
+ handled in `LICENSE`. With the shade filter above, that set is exactly the source release plus
+ `hudi-timeline-server-bundle`; without it, it would be all six bundles. The minified files already carry their
+ upstream copyright headers, which must be preserved. No `NOTICE` changes are required: MIT and ISC do not mandate
+ NOTICE entries, and vis-timeline is taken under its MIT option (ASF policy discourages adding MIT/ISC copyrights to
+ `NOTICE`); were vis-timeline instead taken under Apache-2.0, any upstream `NOTICE` content it ships would have to be
+ propagated.
+- **Spark UI tab (planned follow-up):** Will use existing `spark-core` APIs (`WebUITab`, `WebUIPage`), already provided
Review Comment:
Correct, and this is worth having in the RFC -- it changes the shape of the follow-up, not just its code. `WebUITab`, `WebUIPage`, `WebUI.attachTab`/`detachTab` and `SparkContext.ui` are `private[spark]`; the original bullet calling them "existing spark-core APIs" was misleading.
Rewrote it to record all three consequences:
- The tab must live in an `org.apache.spark` package. Confirmed the precedent you point at -- `hudi-spark-client/src/main/scala/org/apache/spark/HoodieSparkKryoRegistrar.scala`.
- It must be **Scala**, not Java: `WebUIPage.render`'s `Seq[Node]` erases to `scala.collection.Seq` on 2.12 and `scala.collection.immutable.Seq` on 2.13, so no single Java source works for both profiles Hudi cross-builds.
- These internals carry no cross-version stability guarantee. Added the consequence that follows: the tab is a convenience entry point, so dropping it must remain an option that leaves the standalone UI unaffected.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reads the table config / active timeline fresh on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server binds a free port by default
+(`hoodie.embed.timeline.server.port` = `0`), and the bound port is not written back to that property - read it from the
+startup log line (`Started embedded timeline server at <host>:<port>`), or from the resolved
+`FileSystemViewStorageConfig` remote-server port that `EmbeddedTimelineService` propagates it to.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, and each UI request builds a `HoodieTableMetaClient` for its
+own basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **The assets must be explicitly excluded from the engine bundles.** This does *not* come for free.
+ `hudi-timeline-service` is a compile-scope dependency of `hudi-client-common`, and six bundles list
+ `org.apache.hudi:hudi-timeline-service` in their shade `artifactSet`: `hudi-timeline-server-bundle` (intended) plus
+ `hudi-spark-bundle`, `hudi-flink-bundle`, `hudi-utilities-bundle`, `hudi-kafka-connect-bundle` and
+ `hudi-integ-test-bundle`. Their `<filters>` currently exclude only signature files, `META-INF/services/javax.*` and
+ `**/*.proto`, so *any* resource under `src/main/resources/` is unpacked into each shaded JAR. The module has no
+ `src/main/resources` today, which is exactly why this has never bitten us - the ~890KB of vendored assets would be
+ net-new payload in all six bundles, not just the server bundle.
+
+ The first cut is standalone-only, so the UI cannot even be served from an embedded timeline server inside a Spark or
+ Flink driver. Shipping ~890KB of unreachable JS/CSS in five engine bundles is pure bloat, and it would drag the
+ LICENSE obligations below into each of them. The five non-server bundle poms therefore add an explicit shade filter:
+
+ ```xml
+ <filter>
+ <artifact>org.apache.hudi:hudi-timeline-service</artifact>
+ <excludes>
+ <exclude>public/**</exclude>
+ </excludes>
+ </filter>
+ ```
+
+ Only `hudi-timeline-server-bundle` ships the assets. When the embedded/Spark-UI follow-up lands, it must lift this
+ exclusion for the bundles that actually gain an embedded UI (`hudi-spark-bundle`, and `hudi-flink-bundle` if it
+ follows), and add the LICENSE stanzas below to those bundles at the same time. That is a deliberate, reviewable step
+ in the follow-up, not something to inherit silently here.
+- **LICENSE/NOTICE obligations.** Each vendored library needs a "This product bundles ..." stanza in the source-release
+ top-level `LICENSE` **and** in the LICENSE of every bundle whose JAR actually contains the assets, naming the library,
+ its license, and its copyright, with the full MIT/ISC license text inlined the same way existing bundled code is
+ handled in `LICENSE`. With the shade filter above, that set is exactly the source release plus
+ `hudi-timeline-server-bundle`; without it, it would be all six bundles. The minified files already carry their
+ upstream copyright headers, which must be preserved. No `NOTICE` changes are required: MIT and ISC do not mandate
+ NOTICE entries, and vis-timeline is taken under its MIT option (ASF policy discourages adding MIT/ISC copyrights to
+ `NOTICE`); were vis-timeline instead taken under Apache-2.0, any upstream `NOTICE` content it ships would have to be
+ propagated.
+- **Spark UI tab (planned follow-up):** Will use existing `spark-core` APIs (`WebUITab`, `WebUIPage`), already provided
+ in `hudi-spark-client`. No new JARs are added.
+- **No frontend build pipeline.** No npm, webpack, or vite. The JS/CSS files are committed directly and served as-is.
+
+## Additional Quality of Life (QoL) Features to be Considered
+
+The following client-side enhancements were implemented alongside the core features above. These are purely frontend
+additions - no new APIs, Java dependencies, or architectural changes beyond what is specified in this RFC.
+
+- **Bootstrap 5 + renderjson styling** - Bootstrap 5 for layout and renderjson for collapsible, syntax-highlighted JSON
+ in the detail panel.
+- **Instant search and "Go to Now" button** - A search input that filters and focuses the timeline on a specific instant
+ by requested time, plus a button to jump to the latest instant.
+- **Summary statistics bar** - Counts of instants by state (completed, inflight, requested) computed client-side from
+ the loaded data.
+- **Filter controls (state + action)** - Dropdown filters for state and action type using vis-timeline's `DataView`
+ filtering. Users can isolate specific instant types.
+- **Color legend** - Maps colors to instant states for quick reference.
+- **URL `?path=` persistence** - The basepath is stored in the URL query parameter so the view can be bookmarked and
+ shared.
+- **Keyboard navigation** - Left/right arrow keys to move between instants, Escape to close the detail panel.
+- **Help modal** - Pressing `?` opens a modal listing all keyboard shortcuts and UI interactions.
+
+## Future Improvements
+
+Other features we can add later:
+
+1. **Incremental clean range overlay** - A translucent overlay from `lastCompletedCommitTimestamp` to
+ `earliestCommitToRetain` when clean metadata is available, showing which instants are protected from cleaning and
+ which are eligible for removal.
+
+2. **Partition/filegroup write heatmap** - Within one or multiple instants, render a grid colored by write volume from
+ `HoodieCommitMetadata.partitionToWriteStats`. Helps identify skewed partitions or heavily-modified filegroups across
+ commits.
+
+3. **Metadata table timeline overlay** - Render the metadata table's timeline alongside the main table timeline. Useful
+ for diagnosing cases where the metadata table is lagging or has stale entries.
+
+4. **Archived timeline** - Load and render instants from the LSM-based archive files, extending the view beyond the
+ active timeline to include historical actions.
+
+5. **Per-state instant details** - Currently the plan for the UI is to only render the highest state for each instant
+ (e.g., if requested, inflight, and completed all exist, only completed is shown). We should keep this as the default
+ but also let users view the other states' metadata. The simplest approach would be adding state toggle buttons after
+ selecting an instant, so users can fetch and view the JSON content for each state individually.
+
+## Rollout/Adoption Plan
+
+- **Backward compatibility:** Additive change only. All existing `/v1/` endpoints remain unchanged. UI endpoints are
+ under `/v2/hoodie/view/` and the UI entry page at `/ui`. No existing behavior changes.
+- **Release target:** 1.2.x
+- **Documentation:** A new page on hudi.apache.org under the "Operations" section with screenshots, a startup guide, and
+ a debugging walkthrough showing how to use the timeline UI to diagnose common issues (stuck compactions, long-running
+ commits, etc.).
+- **No migration needed:** Net-new feature behind an opt-in flag (`--enable-ui`), no impact on existing users or running
+ systems.
+
+## Test Plan
+
+### Unit Tests
+
+- **`TimelineHandler.getTimelineV2()`**: Verify correct mapping from `HoodieInstant` to `InstantDTOV2`, including:
+ - Every action the active timeline can actually surface is mapped correctly. Note this is *not* one instant per
+ entry in `VALID_ACTIONS_IN_TIMELINE`: the layout filter collapses `(requestedTime, comparableAction)` groups, so
+ a completed `compaction`/`logcompaction`/`clustering` can only ever arrive as `commit`/`deltacommit`/
+ `replacecommit`, and a test that expects a completed instant with action `compaction` will never pass.
+ - `comparableAction` folds the pending actions correctly: a *pending* compaction maps to
+ (`action=compaction`, `comparableAction=commit`); pending logcompaction to `deltacommit`; pending clustering to
+ `replacecommit`. For every other action, `comparableAction` equals `action`.
+ - A compaction that has completed surfaces once, as `action=commit`, `state=COMPLETED` - not as three instants and
+ not with action `compaction`. Assert this against a table with a real completed compaction, since it is the case
+ the group model depends on.
+ - `completionTime` is populated for completed instants and null for requested/inflight instants.
+ - Instants are returned in timeline order.
+- **`TimelineHandler.getInstantDetails()`**: Verify correct Avro deserialization to JSON for various action types
+ (commit metadata, compaction plan, clean plan, etc.).
+- **`getInstantDetails()` path traversal (negative case):** an `instant` param of `../../../../tmp/x` (and similar
Review Comment:
Good catch -- the test as written would have passed against the **unfixed** handler and proved nothing. `getInstantFileName` already resolves a `COMPLETED` instant with a null completion time against the timeline and throws before reaching `StoragePath`, so a `COMPLETED` probe never touches the vulnerable path.
Pinned the negative case to `instantstate=REQUESTED`/`INFLIGHT`, and said why in the test plan so nobody "generalises" it back to all states later.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reads the table config / active timeline fresh on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server binds a free port by default
+(`hoodie.embed.timeline.server.port` = `0`), and the bound port is not written back to that property - read it from the
+startup log line (`Started embedded timeline server at <host>:<port>`), or from the resolved
+`FileSystemViewStorageConfig` remote-server port that `EmbeddedTimelineService` propagates it to.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, and each UI request builds a `HoodieTableMetaClient` for its
+own basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **The assets must be explicitly excluded from the engine bundles.** This does *not* come for free.
+ `hudi-timeline-service` is a compile-scope dependency of `hudi-client-common`, and six bundles list
+ `org.apache.hudi:hudi-timeline-service` in their shade `artifactSet`: `hudi-timeline-server-bundle` (intended) plus
+ `hudi-spark-bundle`, `hudi-flink-bundle`, `hudi-utilities-bundle`, `hudi-kafka-connect-bundle` and
+ `hudi-integ-test-bundle`. Their `<filters>` currently exclude only signature files, `META-INF/services/javax.*` and
+ `**/*.proto`, so *any* resource under `src/main/resources/` is unpacked into each shaded JAR. The module has no
+ `src/main/resources` today, which is exactly why this has never bitten us - the ~890KB of vendored assets would be
+ net-new payload in all six bundles, not just the server bundle.
+
+ The first cut is standalone-only, so the UI cannot even be served from an embedded timeline server inside a Spark or
+ Flink driver. Shipping ~890KB of unreachable JS/CSS in five engine bundles is pure bloat, and it would drag the
+ LICENSE obligations below into each of them. The five non-server bundle poms therefore add an explicit shade filter:
+
+ ```xml
+ <filter>
+ <artifact>org.apache.hudi:hudi-timeline-service</artifact>
+ <excludes>
+ <exclude>public/**</exclude>
+ </excludes>
+ </filter>
+ ```
+
+ Only `hudi-timeline-server-bundle` ships the assets. When the embedded/Spark-UI follow-up lands, it must lift this
+ exclusion for the bundles that actually gain an embedded UI (`hudi-spark-bundle`, and `hudi-flink-bundle` if it
+ follows), and add the LICENSE stanzas below to those bundles at the same time. That is a deliberate, reviewable step
+ in the follow-up, not something to inherit silently here.
+- **LICENSE/NOTICE obligations.** Each vendored library needs a "This product bundles ..." stanza in the source-release
+ top-level `LICENSE` **and** in the LICENSE of every bundle whose JAR actually contains the assets, naming the library,
+ its license, and its copyright, with the full MIT/ISC license text inlined the same way existing bundled code is
+ handled in `LICENSE`. With the shade filter above, that set is exactly the source release plus
+ `hudi-timeline-server-bundle`; without it, it would be all six bundles. The minified files already carry their
+ upstream copyright headers, which must be preserved. No `NOTICE` changes are required: MIT and ISC do not mandate
+ NOTICE entries, and vis-timeline is taken under its MIT option (ASF policy discourages adding MIT/ISC copyrights to
+ `NOTICE`); were vis-timeline instead taken under Apache-2.0, any upstream `NOTICE` content it ships would have to be
+ propagated.
+- **Spark UI tab (planned follow-up):** Will use existing `spark-core` APIs (`WebUITab`, `WebUIPage`), already provided
+ in `hudi-spark-client`. No new JARs are added.
+- **No frontend build pipeline.** No npm, webpack, or vite. The JS/CSS files are committed directly and served as-is.
+
+## Additional Quality of Life (QoL) Features to be Considered
+
+The following client-side enhancements were implemented alongside the core features above. These are purely frontend
+additions - no new APIs, Java dependencies, or architectural changes beyond what is specified in this RFC.
+
+- **Bootstrap 5 + renderjson styling** - Bootstrap 5 for layout and renderjson for collapsible, syntax-highlighted JSON
+ in the detail panel.
+- **Instant search and "Go to Now" button** - A search input that filters and focuses the timeline on a specific instant
+ by requested time, plus a button to jump to the latest instant.
+- **Summary statistics bar** - Counts of instants by state (completed, inflight, requested) computed client-side from
+ the loaded data.
+- **Filter controls (state + action)** - Dropdown filters for state and action type using vis-timeline's `DataView`
+ filtering. Users can isolate specific instant types.
+- **Color legend** - Maps colors to instant states for quick reference.
+- **URL `?path=` persistence** - The basepath is stored in the URL query parameter so the view can be bookmarked and
+ shared.
+- **Keyboard navigation** - Left/right arrow keys to move between instants, Escape to close the detail panel.
+- **Help modal** - Pressing `?` opens a modal listing all keyboard shortcuts and UI interactions.
+
+## Future Improvements
+
+Other features we can add later:
+
+1. **Incremental clean range overlay** - A translucent overlay from `lastCompletedCommitTimestamp` to
+ `earliestCommitToRetain` when clean metadata is available, showing which instants are protected from cleaning and
+ which are eligible for removal.
+
+2. **Partition/filegroup write heatmap** - Within one or multiple instants, render a grid colored by write volume from
+ `HoodieCommitMetadata.partitionToWriteStats`. Helps identify skewed partitions or heavily-modified filegroups across
+ commits.
+
+3. **Metadata table timeline overlay** - Render the metadata table's timeline alongside the main table timeline. Useful
+ for diagnosing cases where the metadata table is lagging or has stale entries.
+
+4. **Archived timeline** - Load and render instants from the LSM-based archive files, extending the view beyond the
+ active timeline to include historical actions.
+
+5. **Per-state instant details** - Currently the plan for the UI is to only render the highest state for each instant
+ (e.g., if requested, inflight, and completed all exist, only completed is shown). We should keep this as the default
+ but also let users view the other states' metadata. The simplest approach would be adding state toggle buttons after
+ selecting an instant, so users can fetch and view the JSON content for each state individually.
+
+## Rollout/Adoption Plan
+
+- **Backward compatibility:** Additive change only. All existing `/v1/` endpoints remain unchanged. UI endpoints are
+ under `/v2/hoodie/view/` and the UI entry page at `/ui`. No existing behavior changes.
+- **Release target:** 1.2.x
+- **Documentation:** A new page on hudi.apache.org under the "Operations" section with screenshots, a startup guide, and
+ a debugging walkthrough showing how to use the timeline UI to diagnose common issues (stuck compactions, long-running
+ commits, etc.).
+- **No migration needed:** Net-new feature behind an opt-in flag (`--enable-ui`), no impact on existing users or running
+ systems.
+
+## Test Plan
+
+### Unit Tests
+
+- **`TimelineHandler.getTimelineV2()`**: Verify correct mapping from `HoodieInstant` to `InstantDTOV2`, including:
+ - Every action the active timeline can actually surface is mapped correctly. Note this is *not* one instant per
+ entry in `VALID_ACTIONS_IN_TIMELINE`: the layout filter collapses `(requestedTime, comparableAction)` groups, so
+ a completed `compaction`/`logcompaction`/`clustering` can only ever arrive as `commit`/`deltacommit`/
+ `replacecommit`, and a test that expects a completed instant with action `compaction` will never pass.
+ - `comparableAction` folds the pending actions correctly: a *pending* compaction maps to
+ (`action=compaction`, `comparableAction=commit`); pending logcompaction to `deltacommit`; pending clustering to
+ `replacecommit`. For every other action, `comparableAction` equals `action`.
+ - A compaction that has completed surfaces once, as `action=commit`, `state=COMPLETED` - not as three instants and
+ not with action `compaction`. Assert this against a table with a real completed compaction, since it is the case
+ the group model depends on.
+ - `completionTime` is populated for completed instants and null for requested/inflight instants.
+ - Instants are returned in timeline order.
+- **`TimelineHandler.getInstantDetails()`**: Verify correct Avro deserialization to JSON for various action types
+ (commit metadata, compaction plan, clean plan, etc.).
+- **`getInstantDetails()` path traversal (negative case):** an `instant` param of `../../../../tmp/x` (and similar
+ `..`-bearing values) must return 404 and must not read outside the table's timeline directory. Plant a readable file
+ at the traversal target and assert its content never appears in the response. A well-formed timestamp that is not in
+ the active timeline must also return 404. This is the assertion the instant-detail route's safety rests on: it can
+ only read instants the timeline actually lists.
+- **Error cases**: Invalid basepath returns an appropriate error. Empty timeline returns an empty list.
+
+### Integration Tests
+
+Following the pattern established by `TestTimelineService.java`:
+
+- Start `TimelineService` with the UI enabled (`--enable-ui`).
+- Create synthetic instants via `HoodieTestUtils` covering multiple action types and states.
+- Verify `GET /v2/hoodie/view/timeline/instants/all` returns the expected list of instants with correct fields.
+- Verify `GET /v2/hoodie/view/timeline/instant` returns valid JSON for a completed instant.
+- Verify `GET /v2/hoodie/view/table/config` and `GET /v2/hoodie/view/table/schema/history` return valid JSON.
+- Verify `GET /ui` returns HTTP 200 with HTML content (static file serving works).
+- **Flag gating (negative case):** start `TimelineService` *without* `--enable-ui` (the default) and assert `GET /ui`,
+ `GET /ui/static/*` and every `GET /v2/hoodie/view/*` route return HTTP 404, while the existing `/v1/` routes still
+ respond. This is the assertion the Scope section's threat model rests on - the widened read surface must not exist
+ unless the flag is explicitly set.
+
+### Bundle Packaging
+
+The `public/**` shade exclusion in [Dependency Impact](#dependency-impact) is silent when it regresses: drop the filter
+and the assets simply reappear in every engine bundle, with nothing failing. Guard it in `packaging/bundle-validation`:
Review Comment:
You are right, and I over-promised. Verified both halves: `ci_run.sh` copies only the flink, kafka-connect, metaserver-server, cli, hadoop-mr, spark, utilities and utilities-slim bundles, so neither `hudi-timeline-server-bundle` (the positive assertion) nor `hudi-integ-test-bundle` (one negative) is in the image; and there is no `jar tf` / `unzip -l` anywhere under `packaging/bundle-validation` -- it is a Docker runtime smoke test with no jar-content primitive.
Took your first option: the RFC now states plainly that this guard is **net-new tooling** -- two bundles wired into that job plus a content-inspection step -- or else needs siting somewhere jar contents are already inspectable. Better to size it here than to discover it during implementation.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
Review Comment:
Fair, and the scaling point is the right one -- up to 1000 reads per request with a Refresh control re-issuing the fetch is a much sharper edge than the single instant-detail read.
Rather than repeat the discipline per call-site, the RFC now states it once: every timeline content read goes through a single shared helper that closes under try-with-resources, covering both the instant-detail route and the per-instant commit-metadata reads in schema-history reconstruction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3568018069
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
Review Comment:
Good catch, both are right. Verified:
- **Timeline:** `/v1`'s `timeline/instants/all` -> `TimelineHandler.getTimeline()` -> `fsView.getTimeline()` = `visibleCommitsAndCompactionTimeline`, i.e. `filterCompletedAndCompactionInstants()` (`FileSystemViewManager:146`) then action-filtered to `{commit, deltacommit, compaction, logcompaction, replacecommit, clustering}` (`BaseHoodieTimeline:196-199`). So `clean`/`rollback`/`savepoint`/`restore`/`indexing` and requested/inflight commit states really are `/v2`-only.
- **Instant detail:** every `/v1` instant route returns `InstantDTO` = `{action, ts, state, requestedTime, completionTime}`. Nothing serves instant content, so there is no counterpart -- and `HoodieCommitMetadata` re-exposes `extraMetadata.schema`, which the paragraph was already counting as widening for schema-history. That was self-contradictory.
Rewrote the threat model to drop the parity claim and list all four `/v2` views as widening the read surface, each with what it adds. Controls (`--enable-ui` off by default, read-only, trusted network) unchanged. Please take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3569328170
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
Review Comment:
Correcting my own justification above -- the conclusion holds but I framed it as a coincidence, and it is stronger than that.
The absence of a `clustering` entry in `InstantComparatorV1`'s map is **by construction, not omission**. In 0.x, clustering and replace-commit shared a filename:
```java
// InstantFileNameGeneratorV1
public String makeRequestedClusteringFileName(String instant) {
// 1n 0.x, clustering and replace commit had the same filename
return makeRequestedReplaceFileName(instant);
}
```
So a v1 clustering instant is written as `.replacecommit.requested` and already carries `action=replacecommit`. There is no `clustering` action in layout v1 at all -- nothing to map, hence no entry.
That makes the pin safe for a better reason than "the maps never disagree in reach": the two maps are **equivalent over everything a v1 timeline can emit**. They agree on `compaction -> commit` and `logcompaction -> deltacommit`, and v2's extra `clustering -> replacecommit` entry is simply inert on v1 input.
RFC updated to say this rather than the weaker extension-based argument.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3569712737
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,879 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
Review Comment:
Not deliberate -- and you are right, it was a mistake. Moved the API to `/ui/api/*`.
Confirmed the premise: `BASE_URL = "/v1/hoodie/view"` is a **private** constant inside `RemoteHoodieTableFileSystemView`. It is not a public REST API at all, it is the RPC namespace of the FSV wire protocol. So `/v2/hoodie/view/*` would have announced itself as *version 2 of that protocol* -- which it is emphatically not -- while squatting on the name a genuine FSV v2 will want. The UI is a different API for a different consumer (a browser, with UI-shaped DTOs); it should not be version-bumping someone else's protocol.
Changes:
- Routes: `/v2/hoodie/view/*` -> `/ui/api/*`, alongside the page (`/ui`) and assets (`/ui/static/*`).
- DTOs: `InstantDTOV2`/`TimelineDTOV2` -> **`UiInstantDTO`/`UiTimelineDTO`** in `dto.ui`. Once the URL is not `/v2`, the `V2` suffix is actively misleading -- it implies a successor to the FSV protocol's `InstantDTO`, which these are not. The `Ui` prefix still solves the simple-name clash that motivated the suffix in the first place.
- Methods: `getTimelineV2` -> `getUiTimeline`, `registerTimelineV2API` -> `registerUiApi`.
One thing this bought that I had not anticipated: it collapses the feature onto **one prefix**. Everything under `/ui` is the UI, and `--enable-ui` gates exactly that subtree. Previously the flag had to gate three unrelated prefixes (`/ui`, `/ui/static/*`, `/v2/hoodie/view/*`) for a single feature, and the flag-gating test had to enumerate all three. It is now a single checkable claim: nothing under `/ui` responds unless the flag is set -- which is also what the threat model rests on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4956748743
+1 on the trim.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4689509122
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14643",
"triggerID" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 672904ed83d9fca3c94e53464b5e22865f8ea6d5 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14643)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4689663012
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14643",
"triggerID" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"triggerType" : "PUSH"
}, {
"hash" : "4e021439627242ed5605c1b376e662ba405c0573",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14646",
"triggerID" : "4e021439627242ed5605c1b376e662ba405c0573",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 672904ed83d9fca3c94e53464b5e22865f8ea6d5 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14643)
* 4e021439627242ed5605c1b376e662ba405c0573 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14646)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3569374791
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
Review Comment:
Both points confirmed, and the second one is the valuable half -- taking it largely dissolves the first.
**The metaClient overload.** Agreed and specified. `FileBasedInternalSchemaStorageManager(HoodieStorage, StoragePath)` (`:61-65`) leaves `metaClient` null, and `getMetaClient()` (`:74-84`) lazily builds a whole second `HoodieTableMetaClient` -- a duplicate `hoodie.properties` read and a duplicate timeline listing on every schema-history request. The RFC now pins the `(HoodieTableMetaClient)` overload (`:67-71`) and passes the request's metaClient.
**The pre-`exists()` walk.** Also correct: `getValidInstants()` runs at `:155`, before `storage.exists(baseSchemaPath)` at `:157`, so a table with no `.schema` dir pays it. But once the manager shares the request's metaClient, that walk is an **in-memory filter over an already-cached active timeline**, not a storage listing -- `getActiveTimeline()` memoizes (`HoodieTableMetaClient:512-516`) and `getCommitsTimeline()` derives from it (`:735-744`). So the residual cost for a no-schema-evolution table is one `exists()` call, not a scan. I fixed the cost model to say exactly that rather than dropping the claim, and did **not** add a handler-side `.schema` pre-check, since with the shared metaClient it would buy nothing.
**One thing worth recording while we are here.** `getHistorySchemaStr()` does read exactly one file -- the `.schema` files are an append-only log, and the newest valid `.schemacommit` already contains every prior version (`SerDeHelper.parseSchemas` -> `TreeMap` by version); `:158-165` sorts the valid names and opens only the last. So the completed-commits walk is **not** history reconstruction -- it is there to discard residual `.schemacommit` files from aborted schema changes, whose timestamps are not completed commits (`:162`).
That distinction is easy to lose, so I put it in the RFC explicitly with a warning not to "optimize" the filter down to a lexicographic max filename -- doing so would resurrect uncommitted schemas.
Please take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3569347357
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
Review Comment:
Correct on all three points. Traced it:
- `getTableSchema()` (`:125-127`) -> `getTableSchema(boolean)` (`:136-138`) -> `getTableSchemaInternal(...).orElseThrow(schemaNotFoundError())`.
- `schemaNotFoundError()` (`:502-504`) -> `new HoodieSchemaNotFoundException("No schema found for table at " + basePath)`.
- `getTableSchemaIfPresent(boolean)` (`:163-165`) returns `getTableSchemaInternal(...)` directly as `Option<HoodieSchema>`, no throw.
So `null` was never reachable, and pointing the UI at a freshly-created table would have come back as a **500**, not the empty-ish 200 the cost model and Error Handling both promised. Switched to `getTableSchemaIfPresent`.
One wrinkle worth noting: there is no no-arg `getTableSchemaIfPresent`, so the call has to pass `metaClient.getTableConfig().populateMetaFields()` explicitly -- which is exactly what the no-arg `getTableSchema()` was supplying on our behalf (`:126`). Specified it that way so the `includeMetadataFields` semantics do not silently change.
Also fixed the two places you pointed at as contradictory, plus a third:
- **Cost model:** now states the no-commits case explicitly -- `currentSchema: null`, `history: []`, a 200 -- and notes that this only holds *because* the resolve goes through `getTableSchemaIfPresent`.
- **Error Handling:** new bullet for an unresolvable table schema (200, not an error), since a brand-new table is a legitimate thing to open the UI against.
- **Test Plan:** a case pinning `getSchemaHistory()` on a commit-less table, so if someone swaps the accessor back the regression is a failing test rather than a 500 in the field.
Please take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3565915925
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
Review Comment:
Done -- flipped the RFC-94 index row from the bare reservation stub to a link to `./rfc-94/rfc-94.md`, matching the RFC-93 / RFC-101 convention. Status left as `UNDER REVIEW`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315][RFC-94] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3401516468
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
+The vis-timeline library is loaded with a two-tier strategy:
+
+1. **Primary:** Load from the `unpkg.com` CDN for automatic patch updates.
+2. **Fallback:** If the CDN is unreachable (e.g., air-gapped environments), load from the bundled
+ copy at `/lib/vis-timeline/`.
+
+The bundled fallback adds ~300KB to the JAR but ensures the UI works without internet access.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`,
+ `clustering`, `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These
+ correspond to the actions in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to
+ `completionTime`. Non-completed instants (requested or inflight) are rendered as point items at
+ `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/timeline/instant/details` and
+ shows the deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths
+since the `FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|--------------------------------|----------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------------------|
+| GET | `/v2/timeline/instants` | `basepath` (required) | `List<InstantDTO>` (v2) | Returns all active instants with requested time, completion time, action, and state |
+| GET | `/v2/timeline/instant/details` | `basepath` (required), `instantTime` (required), `action` (required) | JSON string | Returns the deserialized content of a specific instant's metadata (Avro -> JSON) |
+
+Static files are served from the `/ui/` path, mapped to classpath resources under
+`src/main/resources/public/`.
+
+#### DTO Design
+
+A new v2 `InstantDTO` is introduced in a `v2` package to avoid modifying the existing `/v1/` API
+contract. The existing `InstantDTO` (in `o.a.h.common.table.timeline.dto`) only exposes `action`,
+`timestamp` (requested time), and `state` - it lacks `completionTime`, which is needed for
+rendering range bars on the timeline.
+
+The v2 `InstantDTO` contains:
+
+- `requestedTime` - the instant's requested timestamp (`HoodieInstant.requestedTime()`)
+- `completionTime` - the instant's completion timestamp (`HoodieInstant.getCompletionTime()`), null
+ for non-completed instants
+- `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+- `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+#### Handler Design
+
+A new `UiHandler` class extends `Handler` (following the `InstantStateHandler` pattern) with two
+methods:
+
+1. `getActiveInstants(basePath)` - Retrieves the active timeline via
+ `viewManager.getFileSystemView(basePath).getTimeline()`, iterates over all instants, and maps
+ each `HoodieInstant` to a v2 `InstantDTO` including completion time.
+
+2. `getInstantDetails(basePath, instantTime, action)` - Constructs a `HoodieTableMetaClient` for
+ the given basepath, reads the instant's Avro content via `ActiveTimeline.getInstantDetails()`,
+ and deserializes it to JSON for display in the UI.
+
+#### Registration in RequestHandler
+
+The `UiHandler` is registered in `RequestHandler` following the existing pattern:
+
+- A `UiHandler` field is added alongside the existing handler fields.
+- A `registerUiAPI()` method is added and called from `register()`.
+- Registration is gated behind an `--enable-ui` config flag, same as `enableMarkerRequests` and
+ `enableInstantStateRequests`.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table
+ path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The main feature for the first cut is timeline visualization for a Hudi table.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
Review Comment:
The tab links to a single UI instance, not one tab per table. A Spark app's tables share one embedded server per driver (`EmbeddedTimelineService` tracks the set of basepaths via `basePaths`, calling `addBasePath()` for each table on first write when `hoodie.embed.timeline.server.reuse.enabled` is set), and the backend caches a `FileSystemView` per basepath, so one server serves all tables. The user picks the table inside the UI via the basepath input (persisted in `?path=`). Since the server already knows its set of basepaths, a natural follow-up is to turn that input into a dropdown of registered basepaths. Added a "Multiple Tables in One Application" subsection covering this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315][RFC-94] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3401249650
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
Review Comment:
Good point. Added a threat-model note to the Scope section. Summary:
- The `/v2/` UI endpoints expose the same timeline/filesystem metadata the existing `/v1/` APIs already serve on the same interface, so the UI does not widen the exposure surface.
- It is opt-in and off by default (`--enable-ui` / `hoodie.embed.timeline.server.ui.enable=false`).
- For untrusted networks, operators should front it with a reverse proxy or restrict it to a private interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4689615071
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14643",
"triggerID" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"triggerType" : "PUSH"
}, {
"hash" : "4e021439627242ed5605c1b376e662ba405c0573",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4e021439627242ed5605c1b376e662ba405c0573",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 672904ed83d9fca3c94e53464b5e22865f8ea6d5 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14643)
* 4e021439627242ed5605c1b376e662ba405c0573 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3563433001
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+Two v2 DTOs are introduced in a `v2` package to avoid modifying the existing `/v1/` API contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the v1 `InstantDTO` only exposes `action`, `timestamp`
Review Comment:
You're right -- v1 `InstantDTO.fromInstant` already sets both `requestedTime` and `completionTime` (HUDI-9332), so the "v1 lacks completion time" rationale was wrong. Fixed the DTO Design section.
Keeping v2, but restated the reason honestly: v1 could be consumed directly; the v2 DTOs exist only to give the `/v2/` API a cleaner contract -- `requestTs`/`completionTs` key naming and dropping v1's redundant legacy `ts` field (a duplicate of the requested time the UI doesn't need). Not a new-fields layer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3564129142
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
+The vis-timeline library is loaded with a two-tier strategy:
+
+1. **Primary:** Load from the `unpkg.com` CDN for automatic patch updates.
+2. **Fallback:** If the CDN is unreachable (e.g., air-gapped environments), load from the bundled
+ copy at `/lib/vis-timeline/`.
+
+The bundled fallback adds ~300KB to the JAR but ensures the UI works without internet access.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`,
+ `clustering`, `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These
+ correspond to the actions in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to
+ `completionTime`. Non-completed instants (requested or inflight) are rendered as point items at
+ `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/timeline/instant/details` and
+ shows the deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths
+since the `FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|--------------------------------|----------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------------------|
+| GET | `/v2/timeline/instants` | `basepath` (required) | `List<InstantDTO>` (v2) | Returns all active instants with requested time, completion time, action, and state |
+| GET | `/v2/timeline/instant/details` | `basepath` (required), `instantTime` (required), `action` (required) | JSON string | Returns the deserialized content of a specific instant's metadata (Avro -> JSON) |
+
+Static files are served from the `/ui/` path, mapped to classpath resources under
+`src/main/resources/public/`.
+
+#### DTO Design
+
+A new v2 `InstantDTO` is introduced in a `v2` package to avoid modifying the existing `/v1/` API
+contract. The existing `InstantDTO` (in `o.a.h.common.table.timeline.dto`) only exposes `action`,
+`timestamp` (requested time), and `state` - it lacks `completionTime`, which is needed for
+rendering range bars on the timeline.
+
+The v2 `InstantDTO` contains:
+
+- `requestedTime` - the instant's requested timestamp (`HoodieInstant.requestedTime()`)
+- `completionTime` - the instant's completion timestamp (`HoodieInstant.getCompletionTime()`), null
+ for non-completed instants
+- `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+- `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+#### Handler Design
+
+A new `UiHandler` class extends `Handler` (following the `InstantStateHandler` pattern) with two
+methods:
+
+1. `getActiveInstants(basePath)` - Retrieves the active timeline via
+ `viewManager.getFileSystemView(basePath).getTimeline()`, iterates over all instants, and maps
+ each `HoodieInstant` to a v2 `InstantDTO` including completion time.
+
+2. `getInstantDetails(basePath, instantTime, action)` - Constructs a `HoodieTableMetaClient` for
+ the given basepath, reads the instant's Avro content via `ActiveTimeline.getInstantDetails()`,
+ and deserializes it to JSON for display in the UI.
+
+#### Registration in RequestHandler
+
+The `UiHandler` is registered in `RequestHandler` following the existing pattern:
+
+- A `UiHandler` field is added alongside the existing handler fields.
+- A `registerUiAPI()` method is added and called from `register()`.
+- Registration is gated behind an `--enable-ui` config flag, same as `enableMarkerRequests` and
+ `enableInstantStateRequests`.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table
+ path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The main feature for the first cut is timeline visualization for a Hudi table.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are
+ rendered
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated.
+Completed instants appear as horizontal bars whose width represents duration (requested ->
+completed). Inflight and requested instants appear as point markers. Color indicates state: green
+for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui/`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`).
+The `--enable-ui` flag controls whether the UI static files and `/v2/timeline/` API endpoints are
+registered. When the flag is not set, the timeline server behaves exactly as it does today -
+no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a
+timeline server within the driver process. The UI can be enabled on this embedded server by setting
+a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the
+same UI routes and static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui/`. The embedded server port is logged at startup
+and can also be retrieved from the Spark configuration via the
+`hoodie.embed.timeline.server.port` property.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write
+client initializes and stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a
+"Hudi Timeline" tab is registered in the Spark web UI (typically at `http://localhost:4040`). This
+gives users a single place to discover and access the Hudi Timeline UI without needing to know the
+embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab
+contains a single `WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded
+timeline server at `http://<driver-host>:<timeline-server-port>/ui/`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues
+within the Spark UI shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab,
+providing the complete interactive experience.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration happens in `EmbeddedTimelineService` after the embedded server has started
+successfully. The tab is detached during shutdown to ensure clean cleanup. If the Spark UI is not
+available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement
+
+The Spark UI tab implementation lives in `hudi-spark-client`, not `hudi-timeline-service`, because
+it depends on Spark APIs (`WebUITab`, `WebUIPage`) which are not available in the timeline service
+module. The `EmbeddedTimelineService` (also in `hudi-client-common`) coordinates the registration
+by calling into the Spark-specific tab class when running with a `HoodieSparkEngineContext`.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file
+ serving; no template engine is added.
+- **vis-timeline JS/CSS:** ~300KB bundled as static resources under
+ `src/main/resources/public/lib/vis-timeline/`.
+- **Spark UI tab:** Uses existing `spark-core` APIs (`WebUITab`, `WebUIPage`) which are already
+ provided dependencies in `hudi-spark-client`. No new JARs are added.
+- **No impact on Spark/Flink bundles.** `hudi-timeline-server-bundle` is a separate artifact;
+ adding static resources does not affect engine-specific bundles.
+- **No frontend build pipeline.** No npm, webpack, or vite. The JS/CSS files are committed directly
+ and served as-is.
+
+## Additional Quality of Life (QoL) Features to be Considered
+
+The following client-side enhancements were implemented alongside the core features above. These are
+purely frontend additions - no new APIs, Java dependencies, or architectural changes beyond what is
+specified in this RFC.
+
+- **Bootstrap 5 + renderjson styling** - Bootstrap 5 for layout and renderjson for collapsible,
+ syntax-highlighted JSON in the detail panel.
+- **Instant search and "Go to Now" button** - A search input that filters and focuses the timeline
+ on a specific instant by requested time, plus a button to jump to the latest instant.
+- **Summary statistics bar** - Counts of instants by state (completed, inflight, requested)
+ computed client-side from the loaded data.
+- **Filter controls (state + action)** - Dropdown filters for state and action type using
+ vis-timeline's `DataView` filtering. Users can isolate specific instant types.
+- **Color legend** - Maps colors to instant states for quick reference.
+- **URL `?path=` persistence** - The basepath is stored in the URL query parameter so the view can
+ be bookmarked and shared.
+- **Keyboard navigation** - Left/right arrow keys to move between instants, Escape to close the
+ detail panel.
+- **Help modal** - Pressing `?` opens a modal listing all keyboard shortcuts and UI interactions.
+
+## Future Improvements
+
+Other features we can add later:
+
+1. **Incremental clean range overlay** - A translucent overlay from `lastCompletedCommitTimestamp`
+ to `earliestCommitToRetain` when clean metadata is available, showing which instants are
+ protected from cleaning and which are eligible for removal.
+
+2. **Partition/filegroup write heatmap** - Within one or multiple instants, render a grid colored
+ by write volume from `HoodieCommitMetadata.partitionToWriteStats`. Helps identify skewed
+ partitions or heavily-modified filegroups across commits.
+
+3. **Metadata table timeline overlay** - Render the metadata table's timeline alongside the main
+ table timeline. Useful for diagnosing cases where the metadata table is lagging or has stale
+ entries.
+
+4. **Archived timeline** - Load and render instants from the LSM-based archive files, extending
+ the view beyond the active timeline to include historical actions.
+
+5. **Per-state instant details** - Currently the plan for the UI is to only render the highest state
+ for each instant (e.g., if requested, inflight, and completed all exist, only completed is shown).
+ We should keep this as the default but also let users view the other states' metadata.
+ The simplest approach would be adding state toggle buttons after selecting an instant, so users
+ can fetch and view the JSON content for each state individually.
+
+## Rollout/Adoption Plan
+
+- **Backward compatibility:** Additive change only. All existing `/v1/` endpoints remain unchanged.
+ UI endpoints are under `/v2/timeline/` and static files at `/ui/`. No existing behavior changes.
+- **Release target:** 1.2.x
+- **Documentation:** A new page on hudi.apache.org under the "Operations" section with screenshots,
+ a startup guide, and a debugging walkthrough showing how to use the timeline UI to diagnose common
+ issues (stuck compactions, long-running commits, etc.).
+- **No migration needed:** Net-new feature behind an opt-in flag (`--enable-ui`), no impact on
+ existing users or running systems.
+
+## Test Plan
+
+### Unit Tests
+
+- **`UiHandler.getActiveInstants()`**: Verify correct mapping from `HoodieInstant` to v2
+ `InstantDTO`, including:
+ - All action types in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` are mapped correctly.
+ - `completionTime` is populated for completed instants and null for requested/inflight instants.
+ - Instants are returned in timeline order.
+- **`UiHandler.getInstantDetails()`**: Verify correct Avro deserialization to JSON for various
+ action types (commit metadata, compaction plan, clean plan, etc.).
+- **Error cases**: Invalid basepath returns an appropriate error. Empty timeline returns an empty
+ list.
+
+### Integration Tests
+
Review Comment:
The "Alternatives Considered" section is not in the branch: neither the heading nor its content (standalone React/Node SPA, richer CLI output, a separate UI service/module) appears in rfc-94.md at 4e02143 or at the current head 3b093ed. The only trace is "No frontend build pipeline (npm, webpack, vite) is needed" in Frontend, which states the choice without weighing the rejected options.
Worth adding as its own section - it is the part that explains why static vanilla JS over an SPA, and why extending the CLI was not enough.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3568610848
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
Review Comment:
Verified, this one is correct. `ActiveTimelineV2.getContentStream` -> `readDataStreamFromPath` -> `metaClient.getStorage().open(filePath)` (`ActiveTimelineV2:282-284, 779-785`) hands back a raw open stream the caller owns, while the deprecated `getInstantDetails()` default (`HoodieInstantReader:48-54`) wrapped it in try-with-resources and read to a `byte[]`. Switching to `getContentStream` moves that responsibility onto the handler, and the RFC did not say so.
Added to the `getInstantDetails` handler spec:
> Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself, `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not just on success.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3566707512
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
Review Comment:
🤖 One thing to flag on the switch from `getInstantDetails()` to `getContentStream(instant)`: the latter returns a raw open `InputStream` (`metaClient.getStorage().open(...)`), whereas the deprecated `getInstantDetails()` read into a `byte[]` and closed the stream for you. Could you note that the handler reads this under try-with-resources so the handle is released on both the success and the 500/deserialization-failure paths? Otherwise each instant-detail read leaks an open stream.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. The instant is built with the request's metaClient via `metaClient.getInstantGenerator()`;
Review Comment:
🤖 Here the `HoodieInstant` is built directly from the request's `instant`/`instantaction`/`instantstate` params, but only `state`/`action` are validated (next line). For a requested/inflight instant, `instant` flows unchecked into the filename and then into `new StoragePath(timelinePath, <instant>.<ext>)`, so a value like `../../../../tmp/x` can resolve outside `.hoodie` on HDFS/local FS. @nsivabalan could you confirm whether `instant` should be validated as a well-formed instant time (or matched against the loaded active timeline) before the read, to avoid a path-traversal read?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3565915925
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
Review Comment:
Done -- flipped the row to `| 94 | [Hudi Timeline User Interface (UI)](./rfc-94/rfc-94.md) | :eyes: `'`UNDER REVIEW`'` |`, matching the RFC-93/RFC-101 convention. Status left as `UNDER REVIEW`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3563466648
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
Review Comment:
Both sources. Pinned this down in the RFC (new "Schema-History Reconstruction" subsection):
- `currentSchema` -- `TableSchemaResolver.getTableSchema()`.
- `history` -- walks the completed commits timeline (commit/deltacommit/replacecommit), most recent `limit` instants only (default 200, max 1000), reads each `HoodieCommitMetadata` and takes the schema under `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` key in `extraMetadata`). Deduped: an entry is recorded only when the schema differs from the previous instant's, so runs of same-schema commits collapse to one change entry.
- `internalSchemaHistory` (optional) -- the `.hoodie/.schema/` history string from `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()`, included only when `InternalSchema` is in use.
Cost model: one schema resolve + at most `limit` completed-commit metadata reads (bounded by the active timeline; archived is out of scope) + one `.schema` read. A table that never evolved its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and omits `internalSchemaHistory` -- no extra scanning, no error. Matches the first-cut impl in #13147.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
Review Comment:
Good callout, in my current implementation, we are indeed serving hoodie.properties.
Maybe we should add a note here to not serve the full list of `hoodie.properties` if we want to render it. I'll add that in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315][RFC-94] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3401316160
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
Review Comment:
Agreed, an outbound CDN call is the wrong default for an internal debugging tool. Flipped it: the UI now serves the bundled vis-timeline copy by default and makes no external network calls, so air-gapped and security-conscious setups work out of the box. Dropped the CDN-first strategy; a CDN source can be added later as an opt-in flag if automatic patch updates are ever wanted. Updated the "JavaScript Delivery" section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4688801093
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 672904ed83d9fca3c94e53464b5e22865f8ea6d5 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4688846701
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14643",
"triggerID" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 672904ed83d9fca3c94e53464b5e22865f8ea6d5 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14643)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3569685308
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,879 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as. Equal to `action` for everything
+ except pending `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it
+ server-side keeps the mapping table in one place rather than duplicating it in JavaScript.
+
+ Concretely, this is `InstantComparatorV2.getComparableAction(action)`. Note there is no version-dispatched
+ accessor to call instead: `getComparableAction` is a `public static` on `InstantComparatorV1`/`V2`, the
+ `InstantComparator` interface declares only the three comparator getters, and the one place the mapper is
+ selected by layout version (`TimelineLayout.filterHoodieInstantsByLatestState`) is private.
+
+ Pinning the v2 static is safe for layout-v1 tables because the two maps are *equivalent over everything a v1
+ timeline can emit*. They agree on `compaction -> commit` and `logcompaction -> deltacommit`. The only difference
+ is v2's `clustering -> replacecommit` entry, and v1 has no `clustering` action to apply it to: in 0.x, clustering
+ and replace-commit shared a filename (`InstantFileNameGeneratorV1.makeRequestedClusteringFileName()` delegates to
+ `makeRequestedReplaceFileName()`), so a v1 clustering instant is written as `.replacecommit.requested` and already
+ carries `action=replacecommit`. The mapping is a no-op there, which is exactly why `InstantComparatorV1`'s map has
+ no `clustering` entry - there is nothing to map, not something missing.
+
+ If a second caller ever needs this mapping, lifting `getComparableAction` onto `InstantComparator` (or
+ `TimelineLayout`) is the clean follow-up; doing it here would mean changing a shared `hudi-common` interface for a
+ UI display field, which this RFC does not need.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` via `InstantComparatorV2.getComparableAction(action)` - the same mapping
+ the filter used (see [DTO Design](#dto-design)) - and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ **Stream ownership applies to every timeline content read in this RFC, not just this one.** Unlike the deprecated
+ `getInstantDetails()`, which read into a `byte[]` and closed the stream itself, `getContentStream` hands back a raw
+ open stream (`metaClient.getStorage().open(...)`) that the caller owns. Every such read - this route, and the up-to
+ `limit` per-instant commit-metadata reads in [Schema-History Reconstruction](#schema-history-reconstruction) - goes
+ through a single shared helper that closes under try-with-resources, so the handle is released on the
+ deserialization-failure path and not only on success. The schema-history path is the one that actually matters: at up
+ to 1000 reads per request, with a Refresh control re-issuing the fetch, a leaked handle there exhausts file
+ descriptors quickly.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema, resolved with
+ `TableSchemaResolver.getTableSchemaIfPresent(metaClient.getTableConfig().populateMetaFields())`; `null` when the
+ `Option` is empty.
+
+ Not `getTableSchema()`: that overload ends in `getTableSchemaInternal(...).orElseThrow(schemaNotFoundError())` and
+ raises `HoodieSchemaNotFoundException` on a table with no resolvable schema, so it cannot express "unresolved" as a
+ return value at all. `getTableSchemaIfPresent(boolean)` returns `Option<HoodieSchema>` and does not throw, which is
+ what this view needs - a brand-new table with no commits is a normal thing to point the UI at, not a 500. There is no
+ no-arg `getTableSchemaIfPresent`, so the call passes `populateMetaFields()` explicitly, which is exactly what the
+ no-arg `getTableSchema()` was supplying on our behalf.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
Review Comment:
Agreed, and this one is a response-contract defect rather than a detail, so it goes in the RFC.
The sharp end is the one you identify second: the oldest scanned entry has no predecessor *inside the window*, so under the "differs from the previous instant" rule it is always emitted -- as a **change**. On a table with more than `limit` commits that is a schema change that never happened, surfaced on the one tab whose entire job is saying when the schema changed.
Fixed in the response, not by widening the scan:
- The oldest entry is typed **`baseline`** ("the schema as of the oldest commit scanned"); every later entry is typed `change`.
- Added **`window`**: `{ oldestInstantScanned, truncated }`, where `truncated` means completed commits exist older than the scanned window -- i.e. the baseline is the window edge, not the table's first schema.
The UI can then label the first entry honestly and show that it is looking at a partial history. Without the flag the truncation is silent and not recoverable client-side, which is why it belongs in the contract rather than being left to the implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3568958189
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
+The vis-timeline library is loaded with a two-tier strategy:
+
+1. **Primary:** Load from the `unpkg.com` CDN for automatic patch updates.
+2. **Fallback:** If the CDN is unreachable (e.g., air-gapped environments), load from the bundled
+ copy at `/lib/vis-timeline/`.
+
+The bundled fallback adds ~300KB to the JAR but ensures the UI works without internet access.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`,
+ `clustering`, `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These
+ correspond to the actions in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to
+ `completionTime`. Non-completed instants (requested or inflight) are rendered as point items at
+ `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/timeline/instant/details` and
+ shows the deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths
+since the `FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|--------------------------------|----------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------------------|
+| GET | `/v2/timeline/instants` | `basepath` (required) | `List<InstantDTO>` (v2) | Returns all active instants with requested time, completion time, action, and state |
+| GET | `/v2/timeline/instant/details` | `basepath` (required), `instantTime` (required), `action` (required) | JSON string | Returns the deserialized content of a specific instant's metadata (Avro -> JSON) |
+
+Static files are served from the `/ui/` path, mapped to classpath resources under
+`src/main/resources/public/`.
+
+#### DTO Design
+
+A new v2 `InstantDTO` is introduced in a `v2` package to avoid modifying the existing `/v1/` API
+contract. The existing `InstantDTO` (in `o.a.h.common.table.timeline.dto`) only exposes `action`,
+`timestamp` (requested time), and `state` - it lacks `completionTime`, which is needed for
+rendering range bars on the timeline.
+
+The v2 `InstantDTO` contains:
+
+- `requestedTime` - the instant's requested timestamp (`HoodieInstant.requestedTime()`)
+- `completionTime` - the instant's completion timestamp (`HoodieInstant.getCompletionTime()`), null
+ for non-completed instants
+- `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+- `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+#### Handler Design
+
+A new `UiHandler` class extends `Handler` (following the `InstantStateHandler` pattern) with two
+methods:
+
+1. `getActiveInstants(basePath)` - Retrieves the active timeline via
+ `viewManager.getFileSystemView(basePath).getTimeline()`, iterates over all instants, and maps
+ each `HoodieInstant` to a v2 `InstantDTO` including completion time.
+
+2. `getInstantDetails(basePath, instantTime, action)` - Constructs a `HoodieTableMetaClient` for
+ the given basepath, reads the instant's Avro content via `ActiveTimeline.getInstantDetails()`,
+ and deserializes it to JSON for display in the UI.
+
+#### Registration in RequestHandler
+
+The `UiHandler` is registered in `RequestHandler` following the existing pattern:
+
+- A `UiHandler` field is added alongside the existing handler fields.
+- A `registerUiAPI()` method is added and called from `register()`.
+- Registration is gated behind an `--enable-ui` config flag, same as `enableMarkerRequests` and
+ `enableInstantStateRequests`.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table
+ path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The main feature for the first cut is timeline visualization for a Hudi table.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are
+ rendered
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated.
+Completed instants appear as horizontal bars whose width represents duration (requested ->
+completed). Inflight and requested instants appear as point markers. Color indicates state: green
+for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui/`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`).
+The `--enable-ui` flag controls whether the UI static files and `/v2/timeline/` API endpoints are
+registered. When the flag is not set, the timeline server behaves exactly as it does today -
+no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a
+timeline server within the driver process. The UI can be enabled on this embedded server by setting
+a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the
+same UI routes and static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui/`. The embedded server port is logged at startup
+and can also be retrieved from the Spark configuration via the
+`hoodie.embed.timeline.server.port` property.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write
+client initializes and stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a
+"Hudi Timeline" tab is registered in the Spark web UI (typically at `http://localhost:4040`). This
+gives users a single place to discover and access the Hudi Timeline UI without needing to know the
+embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab
+contains a single `WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded
+timeline server at `http://<driver-host>:<timeline-server-port>/ui/`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues
+within the Spark UI shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab,
+providing the complete interactive experience.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration happens in `EmbeddedTimelineService` after the embedded server has started
+successfully. The tab is detached during shutdown to ensure clean cleanup. If the Spark UI is not
+available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement
+
+The Spark UI tab implementation lives in `hudi-spark-client`, not `hudi-timeline-service`, because
+it depends on Spark APIs (`WebUITab`, `WebUIPage`) which are not available in the timeline service
+module. The `EmbeddedTimelineService` (also in `hudi-client-common`) coordinates the registration
+by calling into the Spark-specific tab class when running with a `HoodieSparkEngineContext`.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file
+ serving; no template engine is added.
+- **vis-timeline JS/CSS:** ~300KB bundled as static resources under
+ `src/main/resources/public/lib/vis-timeline/`.
+- **Spark UI tab:** Uses existing `spark-core` APIs (`WebUITab`, `WebUIPage`) which are already
+ provided dependencies in `hudi-spark-client`. No new JARs are added.
+- **No impact on Spark/Flink bundles.** `hudi-timeline-server-bundle` is a separate artifact;
+ adding static resources does not affect engine-specific bundles.
+- **No frontend build pipeline.** No npm, webpack, or vite. The JS/CSS files are committed directly
+ and served as-is.
+
+## Additional Quality of Life (QoL) Features to be Considered
+
+The following client-side enhancements were implemented alongside the core features above. These are
+purely frontend additions - no new APIs, Java dependencies, or architectural changes beyond what is
+specified in this RFC.
+
+- **Bootstrap 5 + renderjson styling** - Bootstrap 5 for layout and renderjson for collapsible,
+ syntax-highlighted JSON in the detail panel.
+- **Instant search and "Go to Now" button** - A search input that filters and focuses the timeline
+ on a specific instant by requested time, plus a button to jump to the latest instant.
+- **Summary statistics bar** - Counts of instants by state (completed, inflight, requested)
+ computed client-side from the loaded data.
+- **Filter controls (state + action)** - Dropdown filters for state and action type using
+ vis-timeline's `DataView` filtering. Users can isolate specific instant types.
+- **Color legend** - Maps colors to instant states for quick reference.
+- **URL `?path=` persistence** - The basepath is stored in the URL query parameter so the view can
+ be bookmarked and shared.
+- **Keyboard navigation** - Left/right arrow keys to move between instants, Escape to close the
+ detail panel.
+- **Help modal** - Pressing `?` opens a modal listing all keyboard shortcuts and UI interactions.
+
+## Future Improvements
+
+Other features we can add later:
+
+1. **Incremental clean range overlay** - A translucent overlay from `lastCompletedCommitTimestamp`
+ to `earliestCommitToRetain` when clean metadata is available, showing which instants are
+ protected from cleaning and which are eligible for removal.
+
+2. **Partition/filegroup write heatmap** - Within one or multiple instants, render a grid colored
+ by write volume from `HoodieCommitMetadata.partitionToWriteStats`. Helps identify skewed
+ partitions or heavily-modified filegroups across commits.
+
+3. **Metadata table timeline overlay** - Render the metadata table's timeline alongside the main
+ table timeline. Useful for diagnosing cases where the metadata table is lagging or has stale
+ entries.
+
+4. **Archived timeline** - Load and render instants from the LSM-based archive files, extending
+ the view beyond the active timeline to include historical actions.
+
+5. **Per-state instant details** - Currently the plan for the UI is to only render the highest state
+ for each instant (e.g., if requested, inflight, and completed all exist, only completed is shown).
+ We should keep this as the default but also let users view the other states' metadata.
+ The simplest approach would be adding state toggle buttons after selecting an instant, so users
+ can fetch and view the JSON content for each state individually.
+
+## Rollout/Adoption Plan
+
+- **Backward compatibility:** Additive change only. All existing `/v1/` endpoints remain unchanged.
+ UI endpoints are under `/v2/timeline/` and static files at `/ui/`. No existing behavior changes.
+- **Release target:** 1.2.x
+- **Documentation:** A new page on hudi.apache.org under the "Operations" section with screenshots,
+ a startup guide, and a debugging walkthrough showing how to use the timeline UI to diagnose common
+ issues (stuck compactions, long-running commits, etc.).
+- **No migration needed:** Net-new feature behind an opt-in flag (`--enable-ui`), no impact on
+ existing users or running systems.
+
+## Test Plan
+
+### Unit Tests
+
+- **`UiHandler.getActiveInstants()`**: Verify correct mapping from `HoodieInstant` to v2
+ `InstantDTO`, including:
+ - All action types in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` are mapped correctly.
+ - `completionTime` is populated for completed instants and null for requested/inflight instants.
+ - Instants are returned in timeline order.
+- **`UiHandler.getInstantDetails()`**: Verify correct Avro deserialization to JSON for various
+ action types (commit metadata, compaction plan, clean plan, etc.).
+- **Error cases**: Invalid basepath returns an appropriate error. Empty timeline returns an empty
+ list.
+
+### Integration Tests
+
Review Comment:
Still absent at `d908a85`: `grep -i alternativ rfc/rfc-94/rfc-94.md` returns nothing, and no section weighs a rejected option against the chosen one. This is the only item across the last three rounds with no commit behind it.
The text already exists - the 2026-06-12 reply in this thread enumerates the five rejected options (React/Node SPA, richer CLI output, a separate UI service or module, server-side templating, iframe embedding in the Spark UI) with a reason for each. Committing that reply as an "Alternatives Considered" section is the whole fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3569338202
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,784 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** None of the four UI views is `/v1`-parity. The existing `/v1/` routes serve only
+ file-slice/base-file DTOs plus filename-level instant DTOs (action, requested/completion time, state); every `/v2`
+ view widens the read surface beyond that:
+
+ - **Timeline** (`/v2/hoodie/view/timeline/instants/all`) is a superset of `/v1`'s `timeline/instants/all`. The
+ `/v1` route is served from the `FileSystemView`'s write timeline, which is restricted to completed instants plus
+ pending (log)compaction, and to write actions only. The `/v2` route reads the full active timeline, so it
+ additionally exposes `clean`, `rollback`, `savepoint`, `restore` and `indexing` instants, and every
+ requested/inflight state.
+ - **Instant detail** (`/v2/hoodie/view/timeline/instant`) has no `/v1` counterpart at all - no existing route
+ returns instant *content*. It returns deserialized instant metadata, e.g. a `HoodieCommitMetadata` carrying
+ per-partition write stats (file paths, record counts) and the table schema under the `schema` key of
+ `extraMetadata`. Its `instant` param is attacker-controlled and lands in a storage path, so the route resolves
+ the requested instant against the active timeline rather than reconstructing it from the params; see
+ [Handler Design](#handler-design). Without that, it is an arbitrary-path read, not a timeline read.
+ - **Table config** (`/v2/hoodie/view/table/config`) returns the full `hoodie.properties` via
+ `HoodieTableConfig.getProps()`.
+ - **Schema history** (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas - the
+ same schema content the instant-detail view can surface via `extraMetadata`.
+
+ All of it is read-only and already readable by anyone with filesystem access to `.hoodie/`; the UI adds no write or
+ mutation capability, and opens no new network interface (the server binds to all interfaces on the
+ driver/standalone host, as it does today). The most sensitive of the four is table config: properties can reference
+ KMS endpoints, lock-provider connection strings or external key/vault paths, though they rarely embed secrets
+ directly. The first cut serves table config unfiltered (sorted, as-is); a redacting/allowlisted config view is a
+ possible future refinement for less-trusted interfaces. The primary control is that all UI routes are gated behind
+ `--enable-ui` (off by default), with the server assumed to run on a trusted network. Operators on untrusted networks
+ should front the server with a reverse proxy or restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per *comparable* action - `commit`, `deltacommit`, `replacecommit`, `clean`, `rollback`,
+ `savepoint`, `restore`, `indexing`. This is `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` with `compaction`,
+ `logcompaction` and `clustering` folded into the action they complete as, exactly as Hudi itself already does:
+
+ | Pending action | Completes as |
+ |----------------|------------------|
+ | `compaction` | `commit` |
+ | `logcompaction`| `deltacommit` |
+ | `clustering` | `replacecommit` |
+
+ This mapping is not a UI invention - it is `InstantComparatorV2.COMPARABLE_ACTIONS`, and the active timeline has
+ already applied it before the handler sees an instant. `HoodieTableMetaClient.getActiveTimeline()` builds an
+ `ActiveTimelineV2` with the layout filter enabled, which groups instants by `(requestedTime, comparableAction)` and
+ keeps only the highest state. So a completed compaction arrives as a *single* instant whose action is `commit` (its
+ completed file on disk is `<ts>_<ct>.commit`), and a raw `compaction` action only ever reaches the UI while the
+ instant is still pending. Giving `compaction`, `logcompaction` and `clustering` their own rows would therefore
+ produce rows that can never hold a completed instant. Grouping by comparable action instead keeps each row's pending
+ and completed items in the same lane, which is what makes a stalled instant legible (see **Items** below).
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+
+ Items keep their *raw* action for labelling and colouring, so folding rows together does not erase the distinction:
+ a pending compaction is still labelled `compaction` and still sits in the `commit` row. This is precisely what the
+ stuck-compaction diagnosis in [Background](#background) needs - an inflight `compaction` point item with no
+ following range bar in that row is a compaction that never committed. A *completed* compaction does render as an
+ ordinary `commit` range bar, which is correct: on disk and to every other Hudi component, it is a commit. Users who
+ need to confirm the originating operation click the instant; the detail panel shows the `HoodieCommitMetadata`,
+ whose `operationType` reads `COMPACT` (or `CLUSTER`).
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON). The triple must match an instant in the active timeline, else 404 - see [Handler Design](#handler-design) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the instant's raw action (e.g., `commit`, `deltacommit`, `compaction`). Used to label and colour the
+ item, and to address the instant on the `/v2/hoodie/view/timeline/instant` route.
+ - `comparableAction` - the action the instant completes (or would complete) as, derived server-side from the
+ table's `InstantComparator` (`getComparableAction`). Equal to `action` for everything except pending
+ `compaction`/`logcompaction`/`clustering`. This is the vis-timeline *group* key, and computing it server-side
+ keeps the mapping table in one place rather than duplicating it in JavaScript.
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this adds `comparableAction`, renames `requestedTime`/`completionTime` to `requestTs`/`completionTs`, and
+ drops v1's redundant legacy `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+
+ Note that the active timeline has already applied `TimelineLayout.filterHoodieInstantsByLatestState`, which collapses
+ each `(requestedTime, comparableAction)` group down to its highest state. The handler therefore sees at most one
+ instant per logical action, never a requested/inflight/completed triple, and a completed compaction reaches it as a
+ `commit`. This is the behaviour the UI's group model is built around, not something the handler should try to undo:
+ `getTimelineV2` populates `comparableAction` from the same mapping the filter used, and leaves `action` as-is.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. A malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+
+ **The requested instant must be resolved against the loaded active timeline, not reconstructed from the request
+ params.** Building a `HoodieInstant` straight from `instant`/`action`/`state` and handing it to the reader is a
+ path-traversal read: `getContentStream` resolves to
+ `new StoragePath(metaClient.getTimelinePath(), <instant>.<action>.<state>)`, and `StoragePath` runs
+ `URI.normalize()`, which collapses `..` segments. An `instant` of `../../../../tmp/x` therefore resolves cleanly
+ outside `.hoodie` and the handler reads whatever is there. The trailing `.<action>.<state>` suffix and the Avro
+ deserialization narrow what an attacker can actually retrieve, but this is an unvalidated attacker-controlled path
+ and should not ship as one.
+
+ The handler already loads the active timeline for this request, so the fix costs nothing: look the `(instant,
+ action, state)` triple up among the timeline's instants and read *that* `HoodieInstant`, returning 404 when no
+ instant matches. This is strictly stronger than regex-validating the timestamp - it also refuses well-formed
+ timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself
+ lists. The instant generator is then only used for instants that came from the timeline.
+
+ Unlike the deprecated `getInstantDetails()`, which read into a `byte[]` and closed the stream itself,
+ `getContentStream` hands back a raw open stream (`metaClient.getStorage().open(...)`) that the caller owns. The
+ handler must read it under try-with-resources so the handle is released on the deserialization-failure path too, not
+ just on success.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+All four methods - `getTimelineV2`, `getInstantDetails`, `getTableConfig` and `getSchemaHistory` - build a short-lived
+`HoodieTableMetaClient` for the request's basepath, read from it, and discard it; no metaClient is shared across
+Javalin's request threads. At human-click frequency the construction cost is negligible: every route pays one
+`hoodie.properties` read, and all but `getTableConfig` additionally pay one active-timeline listing (`getTimelineV2` to
+map it, `getInstantDetails` to resolve the instant it reads through `getContentStream`, `getSchemaHistory` to walk the
+completed commits). `getInstantDetails` then reads a single instant file's content on top of that. A fresh per-request
+instance is always current and keeps each request's read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
Review Comment:
🤖 `getSchemaHistory` reads up to `limit` (capped at 1000) commit-metadata instants per request. If those per-instant reads go through the same raw `getContentStream` API you flagged for `getInstantDetails`, the same stream-close discipline is needed here — but at up to 1000 reads per request, and with the Refresh control re-issuing the fetch, a leaked handle would exhaust file descriptors far faster than the single-read path. Could you confirm each per-instant read is closed (e.g. via a helper like `readCommitMetadata`, or per-read try-with-resources)?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous merged PR #13152:
URL: https://github.com/apache/hudi/pull/13152
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3566198352
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
+The vis-timeline library is loaded with a two-tier strategy:
+
+1. **Primary:** Load from the `unpkg.com` CDN for automatic patch updates.
+2. **Fallback:** If the CDN is unreachable (e.g., air-gapped environments), load from the bundled
+ copy at `/lib/vis-timeline/`.
+
+The bundled fallback adds ~300KB to the JAR but ensures the UI works without internet access.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`,
+ `clustering`, `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These
+ correspond to the actions in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to
+ `completionTime`. Non-completed instants (requested or inflight) are rendered as point items at
+ `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/timeline/instant/details` and
+ shows the deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths
+since the `FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|--------------------------------|----------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------------------|
+| GET | `/v2/timeline/instants` | `basepath` (required) | `List<InstantDTO>` (v2) | Returns all active instants with requested time, completion time, action, and state |
+| GET | `/v2/timeline/instant/details` | `basepath` (required), `instantTime` (required), `action` (required) | JSON string | Returns the deserialized content of a specific instant's metadata (Avro -> JSON) |
+
+Static files are served from the `/ui/` path, mapped to classpath resources under
+`src/main/resources/public/`.
+
+#### DTO Design
+
+A new v2 `InstantDTO` is introduced in a `v2` package to avoid modifying the existing `/v1/` API
+contract. The existing `InstantDTO` (in `o.a.h.common.table.timeline.dto`) only exposes `action`,
+`timestamp` (requested time), and `state` - it lacks `completionTime`, which is needed for
+rendering range bars on the timeline.
+
+The v2 `InstantDTO` contains:
+
+- `requestedTime` - the instant's requested timestamp (`HoodieInstant.requestedTime()`)
+- `completionTime` - the instant's completion timestamp (`HoodieInstant.getCompletionTime()`), null
+ for non-completed instants
+- `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+- `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+#### Handler Design
+
+A new `UiHandler` class extends `Handler` (following the `InstantStateHandler` pattern) with two
+methods:
+
+1. `getActiveInstants(basePath)` - Retrieves the active timeline via
+ `viewManager.getFileSystemView(basePath).getTimeline()`, iterates over all instants, and maps
+ each `HoodieInstant` to a v2 `InstantDTO` including completion time.
+
+2. `getInstantDetails(basePath, instantTime, action)` - Constructs a `HoodieTableMetaClient` for
+ the given basepath, reads the instant's Avro content via `ActiveTimeline.getInstantDetails()`,
+ and deserializes it to JSON for display in the UI.
+
+#### Registration in RequestHandler
+
+The `UiHandler` is registered in `RequestHandler` following the existing pattern:
+
+- A `UiHandler` field is added alongside the existing handler fields.
+- A `registerUiAPI()` method is added and called from `register()`.
+- Registration is gated behind an `--enable-ui` config flag, same as `enableMarkerRequests` and
+ `enableInstantStateRequests`.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table
+ path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The main feature for the first cut is timeline visualization for a Hudi table.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are
+ rendered
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated.
+Completed instants appear as horizontal bars whose width represents duration (requested ->
+completed). Inflight and requested instants appear as point markers. Color indicates state: green
+for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui/`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`).
+The `--enable-ui` flag controls whether the UI static files and `/v2/timeline/` API endpoints are
+registered. When the flag is not set, the timeline server behaves exactly as it does today -
+no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a
+timeline server within the driver process. The UI can be enabled on this embedded server by setting
+a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the
+same UI routes and static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui/`. The embedded server port is logged at startup
+and can also be retrieved from the Spark configuration via the
+`hoodie.embed.timeline.server.port` property.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write
+client initializes and stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a
+"Hudi Timeline" tab is registered in the Spark web UI (typically at `http://localhost:4040`). This
+gives users a single place to discover and access the Hudi Timeline UI without needing to know the
+embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab
+contains a single `WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded
+timeline server at `http://<driver-host>:<timeline-server-port>/ui/`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues
+within the Spark UI shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab,
+providing the complete interactive experience.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration happens in `EmbeddedTimelineService` after the embedded server has started
+successfully. The tab is detached during shutdown to ensure clean cleanup. If the Spark UI is not
+available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement
+
+The Spark UI tab implementation lives in `hudi-spark-client`, not `hudi-timeline-service`, because
+it depends on Spark APIs (`WebUITab`, `WebUIPage`) which are not available in the timeline service
+module. The `EmbeddedTimelineService` (also in `hudi-client-common`) coordinates the registration
+by calling into the Spark-specific tab class when running with a `HoodieSparkEngineContext`.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file
+ serving; no template engine is added.
+- **vis-timeline JS/CSS:** ~300KB bundled as static resources under
+ `src/main/resources/public/lib/vis-timeline/`.
+- **Spark UI tab:** Uses existing `spark-core` APIs (`WebUITab`, `WebUIPage`) which are already
+ provided dependencies in `hudi-spark-client`. No new JARs are added.
+- **No impact on Spark/Flink bundles.** `hudi-timeline-server-bundle` is a separate artifact;
+ adding static resources does not affect engine-specific bundles.
+- **No frontend build pipeline.** No npm, webpack, or vite. The JS/CSS files are committed directly
+ and served as-is.
+
+## Additional Quality of Life (QoL) Features to be Considered
+
+The following client-side enhancements were implemented alongside the core features above. These are
+purely frontend additions - no new APIs, Java dependencies, or architectural changes beyond what is
+specified in this RFC.
+
+- **Bootstrap 5 + renderjson styling** - Bootstrap 5 for layout and renderjson for collapsible,
+ syntax-highlighted JSON in the detail panel.
+- **Instant search and "Go to Now" button** - A search input that filters and focuses the timeline
+ on a specific instant by requested time, plus a button to jump to the latest instant.
+- **Summary statistics bar** - Counts of instants by state (completed, inflight, requested)
+ computed client-side from the loaded data.
+- **Filter controls (state + action)** - Dropdown filters for state and action type using
+ vis-timeline's `DataView` filtering. Users can isolate specific instant types.
+- **Color legend** - Maps colors to instant states for quick reference.
+- **URL `?path=` persistence** - The basepath is stored in the URL query parameter so the view can
+ be bookmarked and shared.
+- **Keyboard navigation** - Left/right arrow keys to move between instants, Escape to close the
+ detail panel.
+- **Help modal** - Pressing `?` opens a modal listing all keyboard shortcuts and UI interactions.
+
+## Future Improvements
+
+Other features we can add later:
+
+1. **Incremental clean range overlay** - A translucent overlay from `lastCompletedCommitTimestamp`
+ to `earliestCommitToRetain` when clean metadata is available, showing which instants are
+ protected from cleaning and which are eligible for removal.
+
+2. **Partition/filegroup write heatmap** - Within one or multiple instants, render a grid colored
+ by write volume from `HoodieCommitMetadata.partitionToWriteStats`. Helps identify skewed
+ partitions or heavily-modified filegroups across commits.
+
+3. **Metadata table timeline overlay** - Render the metadata table's timeline alongside the main
+ table timeline. Useful for diagnosing cases where the metadata table is lagging or has stale
+ entries.
+
+4. **Archived timeline** - Load and render instants from the LSM-based archive files, extending
+ the view beyond the active timeline to include historical actions.
+
+5. **Per-state instant details** - Currently the plan for the UI is to only render the highest state
+ for each instant (e.g., if requested, inflight, and completed all exist, only completed is shown).
+ We should keep this as the default but also let users view the other states' metadata.
+ The simplest approach would be adding state toggle buttons after selecting an instant, so users
+ can fetch and view the JSON content for each state individually.
+
+## Rollout/Adoption Plan
+
+- **Backward compatibility:** Additive change only. All existing `/v1/` endpoints remain unchanged.
+ UI endpoints are under `/v2/timeline/` and static files at `/ui/`. No existing behavior changes.
+- **Release target:** 1.2.x
+- **Documentation:** A new page on hudi.apache.org under the "Operations" section with screenshots,
+ a startup guide, and a debugging walkthrough showing how to use the timeline UI to diagnose common
+ issues (stuck compactions, long-running commits, etc.).
+- **No migration needed:** Net-new feature behind an opt-in flag (`--enable-ui`), no impact on
+ existing users or running systems.
+
+## Test Plan
+
+### Unit Tests
+
+- **`UiHandler.getActiveInstants()`**: Verify correct mapping from `HoodieInstant` to v2
+ `InstantDTO`, including:
+ - All action types in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` are mapped correctly.
+ - `completionTime` is populated for completed instants and null for requested/inflight instants.
+ - Instants are returned in timeline order.
+- **`UiHandler.getInstantDetails()`**: Verify correct Avro deserialization to JSON for various
+ action types (commit metadata, compaction plan, clean plan, etc.).
+- **Error cases**: Invalid basepath returns an appropriate error. Empty timeline returns an empty
+ list.
+
+### Integration Tests
+
Review Comment:
Still absent at `7b43529c`: the document has no "Alternatives Considered" heading, and none of the rejected options (standalone React/Node SPA, richer CLI output, a separate UI service or module, server-side templating, iframe embedding) are weighed anywhere in it. The thread is marked resolved, and every other item from the last round landed as its own commit, but this one has none behind it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4957703585
With regards to `Alternatives Considered`. We didn't really give too much thought into this other than wanting to keep it lightweight and not adding too much bloat to what we already have.
Given that we already have Javalin, we wanted something on top of it.
That said, the design-level forks that did have a real loser:
| Fork | Chosen | Rejected |
|---|---|---|
| Where the UI lives | Served by the timeline server | Standalone service; CLI-only |
| API surface | Own `/ui/api/*` namespace | Extending `/v2/hoodie/view` |
| JS delivery | Vendored + bundled | CDN load |
| Asset packaging | Server bundle + asset-presence gate | Ship assets in all six engine bundles |
| Spark tab | One UI instance + table selector | One tab per table |
| Schema history | Reconstruct on request over a scanned window | Persist/index it |
The rest of the rationale is implementation-level and already lives inline or in the `A1`-`A10` appendix, so I'd rather not duplicate it into a dedicated section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3562777218
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+Two v2 DTOs are introduced in a `v2` package to avoid modifying the existing `/v1/` API contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the v1 `InstantDTO` only exposes `action`, `timestamp`
+ (requested time), and `state`; it lacks completion time, needed for rendering range bars. The v2 `InstantDTO` has:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` / `getSchemaHistory(basePath, limit)` - serve the table-config and schema-history views.
+ Both reuse a `HoodieTableMetaClient` cached per basepath in a `ConcurrentHashMap` (built once on first access), so
+ repeated requests pay only the targeted read, not metaClient construction.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
Review Comment:
The direction makes this stricter than "worth pinning down": `hudi-spark-client` depends on `hudi-client-common`, not the reverse, so `EmbeddedTimelineService` (in `hudi-client-common`) cannot reference a Spark `WebUITab`/`WebUIPage` class at all - the described "calling into the Spark-specific tab class" will not compile as written. Pick an inversion mechanism and state it: a `ServiceLoader`/SPI interface defined in `hudi-client-common` and implemented in `hudi-spark-client`, reflection on a fully-qualified class name, or a callback registered through `HoodieEngineContext` (which the RFC already hints at). This belongs in the RFC even though the tab is deferred.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+Two v2 DTOs are introduced in a `v2` package to avoid modifying the existing `/v1/` API contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the v1 `InstantDTO` only exposes `action`, `timestamp`
+ (requested time), and `state`; it lacks completion time, needed for rendering range bars. The v2 `InstantDTO` has:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
Review Comment:
The staleness is scoped to the two metaClient-backed views: the timeline and instant-detail endpoints read the server-synced `FileSystemView` timeline, so only `getTableConfig` and `getSchemaHistory` (which read the cached metaClient's `tableConfig`/`activeTimeline`) go stale. `HoodieTableMetaClient` snapshots both at construction and only refreshes on an explicit `reloadTableConfig()` / `reloadActiveTimeline()`. Cheapest fix: reload the metaClient on each config/schema request (both are low-frequency clicks), or attach a short TTL - no need to invalidate the whole cache.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
Review Comment:
The parity argument does not cover this endpoint: none of the `/v1/hoodie/view/*` routes serve `hoodie.properties` - they return file-slice/base-file/timeline DTOs only, and `getTableConfig` is the first route to emit `HoodieTableConfig.getProps()` in full. The schema-history view is likewise net-new exposure. Either narrow the threat model to acknowledge that config/schema widen the surface beyond `/v1`, or gate these two views behind their own flag. A blanket key denylist is likely overkill since table props rarely carry secrets, but the threat-model text should stop claiming `/v1` parity for them.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
Review Comment:
The source is both: iterate completed commit/deltacommit instants and read `HoodieCommitMetadata.SCHEMA_KEY` from extraMetadata, plus `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` from `.hoodie/.schema/` when InternalSchema is present. Worth spelling out in the RFC, along with the cost: the commit scan reads up to `limit` (max 1000) commit-metadata files per request, so pair it with the metaClient-staleness fix and consider a lower default for tables that never evolve schema.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
Review Comment:
Confirmed: mounting the classpath `/public` at the server root reserves `/js`, `/css`, `/lib` against any future route. Javalin's static config takes a hosted path, so serving under `/ui/static` (matching the `/ui` entry page) is a one-line change that keeps the API and UI surfaces from colliding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4938382130
@wombatu-kun this PR for a hudi-ui RFC has been dangling for almost a year can you please help to review it so we can land it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3563443786
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+Two v2 DTOs are introduced in a `v2` package to avoid modifying the existing `/v1/` API contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the v1 `InstantDTO` only exposes `action`, `timestamp`
+ (requested time), and `state`; it lacks completion time, needed for rendering range bars. The v2 `InstantDTO` has:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` / `getSchemaHistory(basePath, limit)` - serve the table-config and schema-history views.
+ Both reuse a `HoodieTableMetaClient` cached per basepath in a `ConcurrentHashMap` (built once on first access), so
+ repeated requests pay only the targeted read, not metaClient construction.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
Review Comment:
Good catch -- you're right, that wouldn't compile. `EmbeddedTimelineService` is in `hudi-client-common` and the dependency only runs `hudi-spark-client` -> `hudi-client-common`, so it can't reference `WebUITab`/`WebUIPage` or even `HoodieSparkEngineContext`.
Reworked "Module Placement" into "Module Placement and Dependency Inversion": registration now goes through a `ServiceLoader` SPI -- `TimelineServerUITabProvider` defined in `hudi-client-common` (engine-agnostic signature), implemented as `SparkTimelineServerUITabProvider` in `hudi-spark-client` and registered via `META-INF/services`. `EmbeddedTimelineService` does `ServiceLoader.load(...)` after the server starts and invokes the provider if present; the Spark impl casts the `HoodieEngineContext` to `HoodieSparkEngineContext` and calls `attachTab()`/`detachTab()`. Mirrors how `HoodieTableConfig` discovers `HoodieTableFormat`, keeping `hudi-client-common` free of any Spark compile-time dep.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3563490654
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+Two v2 DTOs are introduced in a `v2` package to avoid modifying the existing `/v1/` API contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the v1 `InstantDTO` only exposes `action`, `timestamp`
+ (requested time), and `state`; it lacks completion time, needed for rendering range bars. The v2 `InstantDTO` has:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
Review Comment:
Good scoping -- agreed, only `getTableConfig`/`getSchemaHistory` go stale; timeline + instant-detail read the server-synced `FileSystemView`.
Going with reload-on-request, no TTL: each config/schema request reloads the cached metaClient's snapshot under a per-basepath guard (`reloadTableConfig()` / `reloadActiveTimeline()`) before reading. These are click-frequency views, so the reload (one `hoodie.properties` read + one timeline listing) is negligible and always current. Skipped the TTL since at this rate it would only re-add a staleness window for no real saving.
Also adding a **Refresh** button on both tabs as UX sugar -- since the server reloads per request, it just re-issues the fetch. Updated the RFC (Handler Design freshness note + user-actions list).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3568032440
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
Review Comment:
Confirmed, and thanks -- this was a real hole in the group model. `filterHoodieInstantsByLatestState` (`TimelineLayout:133-144`) groups on `(requestedTime, getComparableAction(action))` and keeps the highest state, and `COMPARABLE_ACTIONS` (`InstantComparatorV2:51-57`) maps `compaction -> commit`, `logcompaction -> deltacommit`, `clustering -> replacecommit`. So the `compaction`/`logcompaction`/`clustering` rows could never hold a completed instant, and the Test Plan line was indeed unsatisfiable.
Rather than recovering the originating action, the UI now **follows** the mapping Hudi already applies:
- **Groups are comparable actions.** `compaction` renders in the `commit` row, `logcompaction` in `deltacommit`, `clustering` in `replacecommit`. A completed compaction *is* a commit on disk and to every other component, so rendering it as one is correct, not a loss.
- **Items keep their raw action** for label/colour. `InstantDTOV2` now carries both `action` and `comparableAction` (derived server-side via `getComparableAction`, so the map is not duplicated in JS): group by one, label by the other.
- **Stuck-compaction diagnosis still works** -- and is arguably sharper. A pending compaction survives the filter with `action=compaction` intact, so it is an inflight `compaction` point item sitting in the `commit` row with no following range bar. That is exactly the signal. For a completed one, clicking through shows `HoodieCommitMetadata.operationType = COMPACT`.
- **Test Plan restated** around the collapse: assert pending compaction maps to (`action=compaction`, `comparableAction=commit`), and that a completed compaction surfaces once as `action=commit`, rather than asking for one instant per `VALID_ACTIONS_IN_TIMELINE` entry.
Please take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3568600170
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. The instant is built with the request's metaClient via `metaClient.getInstantGenerator()`;
+ a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+`getTimelineV2`, `getTableConfig` and `getSchemaHistory` each build a short-lived `HoodieTableMetaClient` for the
+request's basepath, read from it, and discard it - no metaClient is shared across Javalin's request threads. At
+human-click frequency the construction cost (one `hoodie.properties` read, plus one active-timeline listing for the
+timeline and schema routes) is negligible, and a fresh per-request instance is always current and keeps each request's
+read self-consistent. A long-lived per-basepath cache is deliberately avoided: the data must be re-read on every request
+anyway, so caching the metaClient would save little while forcing an in-place `reloadActiveTimeline()` /
+`reloadTableConfig()` on a mutable object shared across concurrent same-basepath requests. `HoodieTableMetaClient` marks
+its individual accessors `synchronized`, but a compound reload-then-read is not one critical section, so a concurrent
+reload could swap the snapshot between one request's reload and its read - a fresh instance sidesteps that entirely. A
+TTL is likewise avoided: at this request rate it would save no meaningful work and would only add a staleness window. The
+UI also surfaces a **Refresh** control on the Table Config and Schema History tabs; because the server reads fresh on
+every request, the button simply re-issues the fetch. The timeline view is fresh on every request too; instant details
+are immutable once written and need no refresh.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reads the table config / active timeline fresh on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server binds a free port by default
+(`hoodie.embed.timeline.server.port` = `0`), and the bound port is not written back to that property - read it from the
+startup log line (`Started embedded timeline server at <host>:<port>`), or from the resolved
+`FileSystemViewStorageConfig` remote-server port that `EmbeddedTimelineService` propagates it to.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write client initializes and
+stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+> **Status: deferred to a follow-up.** The Spark UI tab depends on embedded-mode enablement and cross-module Spark APIs,
+> so it is split out of the initial PR for the same reason. The design below is retained for that follow-up.
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a "Hudi Timeline" tab is
+registered in the Spark web UI (typically at `http://localhost:4040`). This gives users a single place to discover and
+access the Hudi Timeline UI without needing to know the embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab contains a single
+`WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded timeline server at
+`http://<driver-host>:<timeline-server-port>/ui`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues within the Spark UI
+shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab, providing the complete interactive
+experience.
+
+### Multiple Tables in One Application
+
+A Spark application can write to multiple Hudi tables. The embedded timeline server is shared across them: when
+`hoodie.embed.timeline.server.reuse.enabled` is set, `EmbeddedTimelineService` keeps a single server per driver and
+tracks the set of basepaths using it (`EmbeddedTimelineService.basePaths`), adding each table on its first write. The
+backend already caches one `FileSystemView` per basepath, and each UI request builds a `HoodieTableMetaClient` for its
+own basepath, so that single server serves every table.
+
+The tab therefore links to a single UI instance rather than registering one tab per table. The user selects which
+table to view inside the UI via the basepath input form (persisted in the `?path=` query parameter). Because the
+server already knows the set of basepaths it serves, a natural follow-up is to pre-populate that input as a dropdown of
+registered basepaths so users pick from known tables instead of typing.
+
+Tab registration is guarded so the "Hudi Timeline" tab is registered once per driver, avoiding duplicate tabs when the
+server is reused across tables.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration is triggered by `EmbeddedTimelineService` after the embedded server has started successfully, through the
+provider SPI described in [Module Placement and Dependency Inversion](#module-placement-and-dependency-inversion) below
+(`EmbeddedTimelineService` itself holds no Spark types). The tab is detached during shutdown to ensure clean cleanup. If
+the Spark UI is not available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement and Dependency Inversion
+
+The Spark UI tab implementation depends on Spark APIs (`WebUITab`, `WebUIPage`), so it must live in `hudi-spark-client`.
+`EmbeddedTimelineService`, however, lives in `hudi-client-common`, and the module dependency runs one way only:
+`hudi-spark-client` depends on `hudi-client-common`, never the reverse. `EmbeddedTimelineService` therefore cannot
+reference the Spark tab class - or even `HoodieSparkEngineContext` - directly; doing so would not compile.
+
+Registration is inverted through a `ServiceLoader` SPI, mirroring how `hudi-common` already discovers
+`HoodieTableFormat` implementations (`ServiceLoader.load(HoodieTableFormat.class)` in `HoodieTableConfig`):
+
+- An engine-agnostic provider interface is defined in `hudi-client-common` (no Spark types on its signature), e.g.:
+
+ ```java
+ // hudi-client-common: org.apache.hudi.client.embedded
+ public interface TimelineServerUITabProvider {
+ void register(HoodieEngineContext context, String serverHost, int serverPort);
+ void unregister();
+ }
+ ```
+
+- `hudi-spark-client` supplies the implementation (`SparkTimelineServerUITabProvider`), registered via
+ `META-INF/services/org.apache.hudi.client.embedded.TimelineServerUITabProvider`. Only this implementation touches
+ Spark APIs: it casts the `HoodieEngineContext` to `HoodieSparkEngineContext`, obtains the `SparkContext`/`SparkUI`,
+ and calls `attachTab()` / `detachTab()` with the `WebUITab`.
+- After the embedded server starts, `EmbeddedTimelineService` runs
+ `ServiceLoader.load(TimelineServerUITabProvider.class)` and invokes the single provider found, if any, passing the
+ started server's host and port. When no provider is on the classpath (non-Spark engines) or the Spark UI is
+ unavailable (handled inside the impl, e.g. `spark.ui.enabled=false`), registration is skipped silently. The loaded
+ provider instance is retained so the matching `unregister()` runs at shutdown.
+
+This keeps `hudi-client-common` free of any Spark compile-time dependency while letting the Spark module supply the tab.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file serving; no template
+ engine is added.
+- **Three vendored frontend libraries (~890KB total)** bundled as static resources under
+ `src/main/resources/public/lib/`. All are ASF Category A licenses and may be redistributed in a release:
+ - **vis-timeline** (`lib/vis-timeline/`, ~575KB) - timeline rendering. Dual-licensed Apache-2.0 OR MIT.
+ - **Bootstrap 5** (`lib/bootstrap/`, ~305KB) - layout and styling. MIT.
+ - **renderjson** (`lib/renderjson/`, ~11KB) - collapsible JSON in the detail panel. ISC.
+- **LICENSE/NOTICE obligations.** Each vendored library needs a "This product bundles ..." stanza in the source-release
+ top-level `LICENSE` (and in the `hudi-timeline-server-bundle` LICENSE, since the assets ship inside that JAR) naming
+ the library, its license, and its copyright, with the full MIT/ISC license text inlined the same way existing bundled
+ code is handled in `LICENSE`. The minified files already carry their upstream copyright headers, which must be
+ preserved. No `NOTICE` changes are required: MIT and ISC do not mandate NOTICE entries, and vis-timeline is taken
+ under its MIT option (ASF policy discourages adding MIT/ISC copyrights to `NOTICE`); were vis-timeline instead taken
+ under Apache-2.0, any upstream `NOTICE` content it ships would have to be propagated.
+- **Spark UI tab (planned follow-up):** Will use existing `spark-core` APIs (`WebUITab`, `WebUIPage`), already provided
+ in `hudi-spark-client`. No new JARs are added.
+- **No impact on Spark/Flink bundles.** `hudi-timeline-server-bundle` is a separate artifact; adding static resources
Review Comment:
Verified, and it is actually one bundle worse than you flagged -- **six** poms list `org.apache.hudi:hudi-timeline-service` in their shade `artifactSet`, not five:
```
$ grep -ln "org.apache.hudi:hudi-timeline-service" packaging/*/pom.xml
packaging/hudi-flink-bundle/pom.xml
packaging/hudi-kafka-connect-bundle/pom.xml
packaging/hudi-utilities-bundle/pom.xml
packaging/hudi-timeline-server-bundle/pom.xml
packaging/hudi-spark-bundle/pom.xml
packaging/hudi-integ-test-bundle/pom.xml
```
`hudi-integ-test-bundle` gets it too. Confirmed the rest as well: `hudi-client-common:54` declares the dep with no `<scope>` (so compile), every `<filters>` block is `*:*` excluding only `META-INF/*.SF|DSA|RSA`, `META-INF/services/javax.*` and `**/*.proto`, and `hudi-timeline-service` has no `src/main/resources` today. So the assets would land in all six.
Went with the explicit-filter option you suggested, since the first cut is standalone-only and the UI is not even reachable from an embedded server -- ~890KB of unserveable JS/CSS in five bundles is pure bloat:
```xml
<filter>
<artifact>org.apache.hudi:hudi-timeline-service</artifact>
<excludes>
<exclude>public/**</exclude>
</excludes>
</filter>
```
Changes:
- **Dependency Impact:** replaced the (wrong) *"No impact on Spark/Flink bundles"* bullet with the real packaging story and the filter above.
- **LICENSE bullet:** now scoped to *"the source release plus every bundle whose JAR actually contains the assets"*, which with the filter is exactly `hudi-timeline-server-bundle` -- and explicitly notes it would be all six without it.
- **Embedded follow-up:** must consciously lift the exclusion for the bundles that gain an embedded UI, and add the LICENSE stanzas there at that time.
- **New Test Plan section:** a `bundle-validation` check asserting the server bundle *contains* `public/**` and the five engine bundles do *not*. Without it this regresses silently -- drop the filter and the assets just quietly reappear.
Please take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4690564414
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14643",
"triggerID" : "672904ed83d9fca3c94e53464b5e22865f8ea6d5",
"triggerType" : "PUSH"
}, {
"hash" : "4e021439627242ed5605c1b376e662ba405c0573",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14646",
"triggerID" : "4e021439627242ed5605c1b376e662ba405c0573",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4e021439627242ed5605c1b376e662ba405c0573 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=14646)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9315][RFC-94] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-3017289416
When do we merge it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9315][RFC-94] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-3503470874
> When do we merge it?
I can bring up a discussion on this on the weekly sync once i'm more or less wrapped up with the unstructured story.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315][RFC-94] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3035645812
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
Review Comment:
🤖 The RFC explicitly marks auth/authz as out of scope, but the embedded mode exposes timeline data (including commit metadata with partition paths, write stats, etc.) on the driver's network interface. Could you add a brief note on what guardrails exist — e.g., binding to localhost only by default, or recommending a reverse proxy? Even a sentence acknowledging the threat model would help reviewers feel comfortable with the "trusted network" assumption.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
Review Comment:
🤖 The CDN-first strategy means the UI makes external network calls by default. In many enterprise/production environments this is unexpected for an internal debugging tool. Would it make sense to default to the bundled copy and let users opt into CDN via a config flag instead? That way air-gapped and security-conscious deployments work out of the box.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
+The vis-timeline library is loaded with a two-tier strategy:
+
+1. **Primary:** Load from the `unpkg.com` CDN for automatic patch updates.
+2. **Fallback:** If the CDN is unreachable (e.g., air-gapped environments), load from the bundled
+ copy at `/lib/vis-timeline/`.
+
+The bundled fallback adds ~300KB to the JAR but ensures the UI works without internet access.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`,
+ `clustering`, `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These
+ correspond to the actions in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to
+ `completionTime`. Non-completed instants (requested or inflight) are rendered as point items at
+ `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/timeline/instant/details` and
+ shows the deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths
+since the `FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|--------------------------------|----------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------------------|
+| GET | `/v2/timeline/instants` | `basepath` (required) | `List<InstantDTO>` (v2) | Returns all active instants with requested time, completion time, action, and state |
+| GET | `/v2/timeline/instant/details` | `basepath` (required), `instantTime` (required), `action` (required) | JSON string | Returns the deserialized content of a specific instant's metadata (Avro -> JSON) |
+
+Static files are served from the `/ui/` path, mapped to classpath resources under
+`src/main/resources/public/`.
+
+#### DTO Design
+
+A new v2 `InstantDTO` is introduced in a `v2` package to avoid modifying the existing `/v1/` API
+contract. The existing `InstantDTO` (in `o.a.h.common.table.timeline.dto`) only exposes `action`,
+`timestamp` (requested time), and `state` - it lacks `completionTime`, which is needed for
+rendering range bars on the timeline.
+
+The v2 `InstantDTO` contains:
Review Comment:
🤖 For `getInstantDetails`, constructing a `HoodieTableMetaClient` per request could be expensive if it involves filesystem listing or metadata loading. Is the MetaClient cached or reused from the `FileSystemViewManager`? If not, this could become a latency bottleneck when users click through multiple instants.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
+The vis-timeline library is loaded with a two-tier strategy:
+
+1. **Primary:** Load from the `unpkg.com` CDN for automatic patch updates.
+2. **Fallback:** If the CDN is unreachable (e.g., air-gapped environments), load from the bundled
+ copy at `/lib/vis-timeline/`.
+
+The bundled fallback adds ~300KB to the JAR but ensures the UI works without internet access.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`,
+ `clustering`, `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These
+ correspond to the actions in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to
+ `completionTime`. Non-completed instants (requested or inflight) are rendered as point items at
+ `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/timeline/instant/details` and
+ shows the deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths
+since the `FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|--------------------------------|----------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------------------|
+| GET | `/v2/timeline/instants` | `basepath` (required) | `List<InstantDTO>` (v2) | Returns all active instants with requested time, completion time, action, and state |
+| GET | `/v2/timeline/instant/details` | `basepath` (required), `instantTime` (required), `action` (required) | JSON string | Returns the deserialized content of a specific instant's metadata (Avro -> JSON) |
+
+Static files are served from the `/ui/` path, mapped to classpath resources under
+`src/main/resources/public/`.
+
+#### DTO Design
+
+A new v2 `InstantDTO` is introduced in a `v2` package to avoid modifying the existing `/v1/` API
+contract. The existing `InstantDTO` (in `o.a.h.common.table.timeline.dto`) only exposes `action`,
+`timestamp` (requested time), and `state` - it lacks `completionTime`, which is needed for
+rendering range bars on the timeline.
+
+The v2 `InstantDTO` contains:
+
+- `requestedTime` - the instant's requested timestamp (`HoodieInstant.requestedTime()`)
+- `completionTime` - the instant's completion timestamp (`HoodieInstant.getCompletionTime()`), null
+ for non-completed instants
+- `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+- `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+#### Handler Design
+
+A new `UiHandler` class extends `Handler` (following the `InstantStateHandler` pattern) with two
+methods:
+
+1. `getActiveInstants(basePath)` - Retrieves the active timeline via
+ `viewManager.getFileSystemView(basePath).getTimeline()`, iterates over all instants, and maps
+ each `HoodieInstant` to a v2 `InstantDTO` including completion time.
+
+2. `getInstantDetails(basePath, instantTime, action)` - Constructs a `HoodieTableMetaClient` for
+ the given basepath, reads the instant's Avro content via `ActiveTimeline.getInstantDetails()`,
+ and deserializes it to JSON for display in the UI.
+
+#### Registration in RequestHandler
+
+The `UiHandler` is registered in `RequestHandler` following the existing pattern:
+
+- A `UiHandler` field is added alongside the existing handler fields.
+- A `registerUiAPI()` method is added and called from `register()`.
+- Registration is gated behind an `--enable-ui` config flag, same as `enableMarkerRequests` and
+ `enableInstantStateRequests`.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table
+ path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The main feature for the first cut is timeline visualization for a Hudi table.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
Review Comment:
🤖 For the Spark UI tab integration — how does this work with multiple Hudi tables being written in the same Spark application? Does the tab show a list of all registered basepaths, or does it link to a single UI instance where the user then selects a basepath?
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
+The vis-timeline library is loaded with a two-tier strategy:
+
+1. **Primary:** Load from the `unpkg.com` CDN for automatic patch updates.
+2. **Fallback:** If the CDN is unreachable (e.g., air-gapped environments), load from the bundled
+ copy at `/lib/vis-timeline/`.
+
+The bundled fallback adds ~300KB to the JAR but ensures the UI works without internet access.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`,
+ `clustering`, `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These
+ correspond to the actions in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to
+ `completionTime`. Non-completed instants (requested or inflight) are rendered as point items at
+ `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/timeline/instant/details` and
+ shows the deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths
+since the `FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|--------------------------------|----------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------------------|
+| GET | `/v2/timeline/instants` | `basepath` (required) | `List<InstantDTO>` (v2) | Returns all active instants with requested time, completion time, action, and state |
+| GET | `/v2/timeline/instant/details` | `basepath` (required), `instantTime` (required), `action` (required) | JSON string | Returns the deserialized content of a specific instant's metadata (Avro -> JSON) |
+
+Static files are served from the `/ui/` path, mapped to classpath resources under
+`src/main/resources/public/`.
+
+#### DTO Design
+
+A new v2 `InstantDTO` is introduced in a `v2` package to avoid modifying the existing `/v1/` API
+contract. The existing `InstantDTO` (in `o.a.h.common.table.timeline.dto`) only exposes `action`,
+`timestamp` (requested time), and `state` - it lacks `completionTime`, which is needed for
+rendering range bars on the timeline.
+
+The v2 `InstantDTO` contains:
+
+- `requestedTime` - the instant's requested timestamp (`HoodieInstant.requestedTime()`)
+- `completionTime` - the instant's completion timestamp (`HoodieInstant.getCompletionTime()`), null
+ for non-completed instants
+- `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+- `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+#### Handler Design
+
+A new `UiHandler` class extends `Handler` (following the `InstantStateHandler` pattern) with two
+methods:
+
+1. `getActiveInstants(basePath)` - Retrieves the active timeline via
+ `viewManager.getFileSystemView(basePath).getTimeline()`, iterates over all instants, and maps
+ each `HoodieInstant` to a v2 `InstantDTO` including completion time.
+
+2. `getInstantDetails(basePath, instantTime, action)` - Constructs a `HoodieTableMetaClient` for
+ the given basepath, reads the instant's Avro content via `ActiveTimeline.getInstantDetails()`,
+ and deserializes it to JSON for display in the UI.
+
+#### Registration in RequestHandler
+
+The `UiHandler` is registered in `RequestHandler` following the existing pattern:
+
+- A `UiHandler` field is added alongside the existing handler fields.
+- A `registerUiAPI()` method is added and called from `register()`.
+- Registration is gated behind an `--enable-ui` config flag, same as `enableMarkerRequests` and
+ `enableInstantStateRequests`.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table
+ path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The main feature for the first cut is timeline visualization for a Hudi table.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are
+ rendered
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated.
+Completed instants appear as horizontal bars whose width represents duration (requested ->
+completed). Inflight and requested instants appear as point markers. Color indicates state: green
+for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui/`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`).
+The `--enable-ui` flag controls whether the UI static files and `/v2/timeline/` API endpoints are
+registered. When the flag is not set, the timeline server behaves exactly as it does today -
+no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a
+timeline server within the driver process. The UI can be enabled on this embedded server by setting
+a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the
+same UI routes and static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui/`. The embedded server port is logged at startup
+and can also be retrieved from the Spark configuration via the
+`hoodie.embed.timeline.server.port` property.
+
+#### Starting from a Spark application (driver)
+
+Set the property programmatically on `HoodieWriteConfig` before creating the write client:
+
+```java
+HoodieWriteConfig config = HoodieWriteConfig.newBuilder()
+ .withPath(basePath)
+ .withEmbeddedTimelineServerEnabled(true)
+ .withEmbeddedTimelineServerUIEnabled(true) // enables UI on embedded server
+ // ... other configs
+ .build();
+```
+
+The UI is available for the lifetime of the `EmbeddedTimelineService` - it starts when the write
+client initializes and stops when the client or `SparkContext` is closed.
+
+## Spark UI Tab Integration
+
+When the `EmbeddedTimelineService` starts with the UI enabled inside a Spark application, a
+"Hudi Timeline" tab is registered in the Spark web UI (typically at `http://localhost:4040`). This
+gives users a single place to discover and access the Hudi Timeline UI without needing to know the
+embedded server's port.
+
+### Approach
+
+A custom class extending Spark's `WebUITab` is added to the `hudi-spark-client` module. The tab
+contains a single `WebUIPage` that renders a link to the Hudi Timeline UI running on the embedded
+timeline server at `http://<driver-host>:<timeline-server-port>/ui/`.
+
+The link approach is chosen over embedding the UI in an iframe to avoid layout and scrolling issues
+within the Spark UI shell. Clicking the link opens the full Hudi Timeline UI in a new browser tab,
+providing the complete interactive experience.
+
+### Registration and Lifecycle
+
+```
+SparkContext starts
+ └─> EmbeddedTimelineService starts (with UI enabled)
+ └─> HudiTimelineTab registered via SparkUI.attachTab()
+
+SparkContext stops / write client closes
+ └─> EmbeddedTimelineService stops
+ └─> HudiTimelineTab detached via SparkUI.detachTab()
+```
+
+Registration happens in `EmbeddedTimelineService` after the embedded server has started
+successfully. The tab is detached during shutdown to ensure clean cleanup. If the Spark UI is not
+available (e.g., `spark.ui.enabled=false`), the tab registration is skipped silently.
+
+### Module Placement
+
+The Spark UI tab implementation lives in `hudi-spark-client`, not `hudi-timeline-service`, because
+it depends on Spark APIs (`WebUITab`, `WebUIPage`) which are not available in the timeline service
+module. The `EmbeddedTimelineService` (also in `hudi-client-common`) coordinates the registration
+by calling into the Spark-specific tab class when running with a `HoodieSparkEngineContext`.
+
+## Dependency Impact
+
+- **Zero new Java compile-time dependencies.** The frontend uses Javalin's built-in static file
+ serving; no template engine is added.
+- **vis-timeline JS/CSS:** ~300KB bundled as static resources under
+ `src/main/resources/public/lib/vis-timeline/`.
+- **Spark UI tab:** Uses existing `spark-core` APIs (`WebUITab`, `WebUIPage`) which are already
+ provided dependencies in `hudi-spark-client`. No new JARs are added.
+- **No impact on Spark/Flink bundles.** `hudi-timeline-server-bundle` is a separate artifact;
+ adding static resources does not affect engine-specific bundles.
+- **No frontend build pipeline.** No npm, webpack, or vite. The JS/CSS files are committed directly
+ and served as-is.
+
+## Additional Quality of Life (QoL) Features to be Considered
+
+The following client-side enhancements were implemented alongside the core features above. These are
+purely frontend additions - no new APIs, Java dependencies, or architectural changes beyond what is
+specified in this RFC.
+
+- **Bootstrap 5 + renderjson styling** - Bootstrap 5 for layout and renderjson for collapsible,
+ syntax-highlighted JSON in the detail panel.
+- **Instant search and "Go to Now" button** - A search input that filters and focuses the timeline
+ on a specific instant by requested time, plus a button to jump to the latest instant.
+- **Summary statistics bar** - Counts of instants by state (completed, inflight, requested)
+ computed client-side from the loaded data.
+- **Filter controls (state + action)** - Dropdown filters for state and action type using
+ vis-timeline's `DataView` filtering. Users can isolate specific instant types.
+- **Color legend** - Maps colors to instant states for quick reference.
+- **URL `?path=` persistence** - The basepath is stored in the URL query parameter so the view can
+ be bookmarked and shared.
+- **Keyboard navigation** - Left/right arrow keys to move between instants, Escape to close the
+ detail panel.
+- **Help modal** - Pressing `?` opens a modal listing all keyboard shortcuts and UI interactions.
+
+## Future Improvements
+
+Other features we can add later:
+
+1. **Incremental clean range overlay** - A translucent overlay from `lastCompletedCommitTimestamp`
+ to `earliestCommitToRetain` when clean metadata is available, showing which instants are
+ protected from cleaning and which are eligible for removal.
+
+2. **Partition/filegroup write heatmap** - Within one or multiple instants, render a grid colored
+ by write volume from `HoodieCommitMetadata.partitionToWriteStats`. Helps identify skewed
+ partitions or heavily-modified filegroups across commits.
+
+3. **Metadata table timeline overlay** - Render the metadata table's timeline alongside the main
+ table timeline. Useful for diagnosing cases where the metadata table is lagging or has stale
+ entries.
+
+4. **Archived timeline** - Load and render instants from the LSM-based archive files, extending
+ the view beyond the active timeline to include historical actions.
+
+5. **Per-state instant details** - Currently the plan for the UI is to only render the highest state
+ for each instant (e.g., if requested, inflight, and completed all exist, only completed is shown).
+ We should keep this as the default but also let users view the other states' metadata.
+ The simplest approach would be adding state toggle buttons after selecting an instant, so users
+ can fetch and view the JSON content for each state individually.
+
+## Rollout/Adoption Plan
+
+- **Backward compatibility:** Additive change only. All existing `/v1/` endpoints remain unchanged.
+ UI endpoints are under `/v2/timeline/` and static files at `/ui/`. No existing behavior changes.
+- **Release target:** 1.2.x
+- **Documentation:** A new page on hudi.apache.org under the "Operations" section with screenshots,
+ a startup guide, and a debugging walkthrough showing how to use the timeline UI to diagnose common
+ issues (stuck compactions, long-running commits, etc.).
+- **No migration needed:** Net-new feature behind an opt-in flag (`--enable-ui`), no impact on
+ existing users or running systems.
+
+## Test Plan
+
+### Unit Tests
+
+- **`UiHandler.getActiveInstants()`**: Verify correct mapping from `HoodieInstant` to v2
+ `InstantDTO`, including:
+ - All action types in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE` are mapped correctly.
+ - `completionTime` is populated for completed instants and null for requested/inflight instants.
+ - Instants are returned in timeline order.
+- **`UiHandler.getInstantDetails()`**: Verify correct Avro deserialization to JSON for various
+ action types (commit metadata, compaction plan, clean plan, etc.).
+- **Error cases**: Invalid basepath returns an appropriate error. Empty timeline returns an empty
+ list.
+
+### Integration Tests
+
Review Comment:
🤖 The RFC doesn't have an explicit "Alternatives Considered" section. It would strengthen the proposal to briefly discuss why, for example, a standalone React/Node app was rejected in favor of static JS, or why extending the existing CLI with richer output wasn't sufficient. Even a short paragraph would help.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
+The vis-timeline library is loaded with a two-tier strategy:
+
+1. **Primary:** Load from the `unpkg.com` CDN for automatic patch updates.
+2. **Fallback:** If the CDN is unreachable (e.g., air-gapped environments), load from the bundled
+ copy at `/lib/vis-timeline/`.
+
+The bundled fallback adds ~300KB to the JAR but ensures the UI works without internet access.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`,
+ `clustering`, `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These
+ correspond to the actions in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to
+ `completionTime`. Non-completed instants (requested or inflight) are rendered as point items at
+ `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/timeline/instant/details` and
+ shows the deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths
+since the `FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|--------------------------------|----------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------------------|
+| GET | `/v2/timeline/instants` | `basepath` (required) | `List<InstantDTO>` (v2) | Returns all active instants with requested time, completion time, action, and state |
+| GET | `/v2/timeline/instant/details` | `basepath` (required), `instantTime` (required), `action` (required) | JSON string | Returns the deserialized content of a specific instant's metadata (Avro -> JSON) |
+
Review Comment:
🤖 For tables with thousands of active instants, returning all of them in a single `GET /v2/timeline/instants` response could be expensive both server-side and for the browser to render. Have you considered adding optional pagination or a time-range filter (`from`/`to` query params) to the API? Even if the first cut skips it, documenting the intent would be helpful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #13152:
URL: https://github.com/apache/hudi/pull/13152#issuecomment-4956361017
All open threads are now addressed. Before the next round, I want to flag something about **altitude**, because I think we have collectively drifted and I am as responsible as anyone -- I answered every comment at the depth it was asked, and the document is worse for it.
This RFC has grown by about a third during review, and most of that growth is not design. It is code review of code that does not exist yet: which `public static` to call, which constructor overload, whether a specific stream is closed, how many `exists()` calls a request makes. Several of those notes are *correct* and I have taken them -- but a reader coming to this RFC to evaluate **whether we should build a Hudi timeline UI, and roughly how** now has to wade through `InstantComparatorV1`-vs-`V2` map equivalence to get there. That defeats the purpose of the document.
I would propose we sort review comments into two buckets:
**Belongs in the RFC** -- things that change what we build or what we promise, and that are expensive to reverse once code lands:
- Scope and threat model (the `/v1`-parity claim was simply false; the instant-detail route was an arbitrary-path read).
- The data model (grouping by comparable action -- the original group model had rows that could never hold a completed instant).
- Packaging and licensing (~890KB into five release artifacts, plus ASF LICENSE obligations).
- Feasibility and risk of deferred work (the Spark UI tab rests on `private[spark]` internals; the packaging guard is net-new CI tooling).
**Belongs in PR review** -- things that only change how the code is typed, and that a compiler, a test, or a reviewer will catch anyway:
- Exact method/overload selection.
- Stream-closing discipline at a given call-site.
- Per-call cost accounting to the granularity of individual `exists()` calls.
The second bucket is not unimportant -- it is just cheap to fix later and expensive to specify now, and specifying it now creates a doc that goes stale the moment the implementation deviates by one line.
Concretely, I would like to **trim the implementation minutiae back out** before this merges, keeping the design decisions and the contracts (empty table returns 200; instant reads resolve against the timeline; assets do not ship in engine bundles) and dropping the API-level forensics that justify them. Happy to do that as a follow-up commit on this branch if reviewers agree.
@wombatu-kun -- to be clear, this is not a complaint about your review. You read the actual code rather than the prose, and that is exactly why you caught a path-traversal read, a broken group model, and ~890KB of assets silently landing in five bundles. Those were all worth catching at RFC time. I would just rather bank the design consequences here and take the API-level details to the implementation PR, where they belong and where they can be verified by a compiler instead of by a reader.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3565783951
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+Both `getTableConfig` and `getSchemaHistory` reuse a `HoodieTableMetaClient` cached per basepath in a
+`ConcurrentHashMap`, so repeated requests skip full metaClient construction. A `HoodieTableMetaClient` snapshots its
+`tableConfig` and `activeTimeline` at construction and refreshes them only on demand, so a build-once cache would serve
+stale config/schema for the lifetime of a long-lived embedded driver. To avoid that, each config/schema request first
+reloads the relevant snapshot under a per-basepath guard - `reloadTableConfig()` for `getTableConfig`,
Review Comment:
🤖 The cached `HoodieTableMetaClient` is one mutable object shared across Javalin's request threads, yet each config/schema request now calls `reloadTableConfig()`/`reloadActiveTimeline()` on it before reading. `HoodieTableMetaClient` isn't thread-safe, so two concurrent requests for the same basepath can have one reload swap `tableConfig`/`activeTimeline` while another is mid-read. Does the per-basepath guard span both endpoints and wrap reload+read as one critical section, or could a fresh short-lived metaClient per request (equally cheap at click frequency) avoid the shared mutable state altogether? @nsivabalan could you sanity-check the thread-safety here?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3568637637
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,662 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ Meta["HoodieTableMetaClient\n(active timeline, config, schema)"]
+
+ API --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading from a short-lived
+ `HoodieTableMetaClient` built for the request's basepath - its `getActiveTimeline()` for the timeline routes, and
+ table config/schema for the config/schema routes - and returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs that serve the UI's timeline, config and schema metadata, reading each table
+through a short-lived `HoodieTableMetaClient` built per request (see [Handler Design](#handler-design)).
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTOV2`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTOV2>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+ Both v2 DTOs carry the `V2` suffix to mirror the repo's versioned-class convention (`versioning/v1`/`v2` with
+ `InstantGeneratorV1`/`V2`, `BaseTimelineV1`/`V2`) and to avoid a simple-name clash with the existing `dto.InstantDTO`
+ that `TimelineHandler` still imports for the v1 routes.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `getActiveTimeline()` from the request's `HoodieTableMetaClient` to a
+ `TimelineDTOV2`. The active timeline carries every `VALID_ACTIONS_IN_TIMELINE` action in all states
+ (requested/inflight/completed), which the vis-timeline groups and requested/inflight point items require. The
+ `FileSystemView` timeline (`getFileSystemView(basePath).getTimeline()`) cannot be used here: it is the write timeline
+ filtered to completed plus (log)compaction instants, so it drops `clean`/`rollback`/`savepoint`/`restore`/`indexing`
+ and every requested/inflight state.
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getContentStream(instant)` (the non-deprecated reader method; `getInstantDetails()` is `@Deprecated`) and
+ deserializes it to JSON. The instant is built with the request's metaClient via `metaClient.getInstantGenerator()`;
Review Comment:
Verified -- this is real, and it is the most serious thing found on the RFC so far. Traced it end to end:
- `getContentStream(instant)` -> `getInstantFileNamePath(getInstantFileName(instant))` -> `new StoragePath(metaClient.getTimelinePath(), "<instant>.<action>.<state>")` (`ActiveTimelineV2:282-284, 649-651`).
- `StoragePath` runs `URI.normalize()` (`StoragePath.java:82`), which **collapses `..` segments**. So `instant=../../../../tmp/x` resolves cleanly outside `.hoodie` and the handler opens whatever is there.
The trailing `.<action>.<state>` suffix and the Avro deserialization narrow what is actually retrievable, so it is not a general file-read primitive -- but it is an unvalidated attacker-controlled path and should not ship as one.
Fixed by resolving rather than reconstructing. The handler **already loads the active timeline for this request**, so the check is free: look the `(instant, action, state)` triple up among the timeline's instants, read *that* `HoodieInstant`, and return **404** when nothing matches.
That is deliberately stronger than regex-validating the timestamp, which is the obvious fix and the weaker one: resolution also refuses well-formed timestamps that simply do not exist in the table, so the route can only ever read instants the timeline itself lists. There is no path for a caller-supplied string to reach `StoragePath` at all.
Also:
- API table now documents the 404.
- Threat model cross-links it -- without this the instant-detail view is an arbitrary-path read, not a timeline read, which materially changes that section.
- Test Plan gains a negative case: plant a readable file at the traversal target, assert 404 and that its content never appears in the response.
Please take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3565970548
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,644 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The timeline and instant-detail views are `/v1`-parity - they read the same active-timeline and
+ filesystem metadata the existing `/v1/` REST APIs already serve, on the same network interface (the server binds to
+ all interfaces on the driver/standalone host). Two views widen the read surface beyond `/v1`, whose routes serve only
+ file-slice/base-file/timeline DTOs: the table-config view (`/v2/hoodie/view/table/config`) returns the full
+ `hoodie.properties` via `HoodieTableConfig.getProps()`, and the schema-history view
+ (`/v2/hoodie/view/table/schema/history`) exposes current and historical table schemas. Table properties can reference
+ sensitive material - KMS endpoints, lock-provider connection strings, external key/vault paths - though they rarely
+ embed secrets directly. The first cut serves table config unfiltered (sorted, as-is); the same content is already
+ readable by anyone with filesystem access to `.hoodie/hoodie.properties`. The primary control is that all UI routes,
+ including these two, are gated behind `--enable-ui` (off by default), with the server assumed to run on a trusted
+ network; a redacting/allowlisted config view is a possible future refinement for less-trusted interfaces. The UI adds
+ no write or mutation capability. Operators on untrusted networks should front the server with a reverse proxy or
+ restrict it to a private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui entry + /ui/static/*\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves JavaScript, CSS, and library assets from the classpath
+ (`src/main/resources/public/`) under the `/ui/static/` URL prefix; `UiHandler` serves `index.html` at `/ui`. No
+ server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. Three
+libraries are vendored as static assets: vis-timeline (timeline rendering), Bootstrap 5 (layout/styling), and renderjson
+(collapsible JSON in the detail panel).
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/ # Vendored third-party assets (see Dependency Impact)
+ ├── vis-timeline/ # Timeline rendering (Apache-2.0 OR MIT)
+ │ ├── vis-timeline-graph2d.min.js
+ │ └── vis-timeline-graph2d.min.css
+ ├── bootstrap/ # Layout/styling (MIT)
+ │ ├── bootstrap.bundle.min.js
+ │ └── bootstrap.min.css
+ └── renderjson/ # Collapsible JSON detail panel (ISC)
+ └── renderjson.js
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+All three libraries are served from bundled copies under `/ui/static/lib/` (`/ui/static/lib/vis-timeline/`,
+`/ui/static/lib/bootstrap/`, `/ui/static/lib/renderjson/`). The UI makes no external network calls, so it works out of
+the box in air-gapped and security-conscious deployments with no extra configuration. The bundled, minified assets add
+~890KB to the JAR (vis-timeline ~575KB, Bootstrap 5 ~305KB, renderjson ~11KB).
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current schema, per-commit schema-change history from the last `limit` commits, and `.schema` internal-schema history when present |
+
+Static assets (JS, CSS, library files) are served from the classpath directory `src/main/resources/public/`, mounted
+under the `/ui/static/` URL prefix via Javalin's static-files `hostedPath` (e.g., `/ui/static/js/timeline.js`,
+`/ui/static/lib/...`). Namespacing everything UI under `/ui` keeps the UI surface from colliding with `/v1/`, `/v2/`, or
+any future module-registered routes on the same Javalin instance, rather than reserving root prefixes like `/js`,
+`/css`, and `/lib`. `UiHandler` additionally registers `GET /ui`, which returns `index.html` (with asset links pointing
+at `/ui/static/...`) to give the UI a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+The UI's timeline endpoint returns a `TimelineDTOV2` built from two v2 DTOs in a `v2` package, leaving the existing
+`/v1/` API contract untouched.
+
+The v1 `InstantDTO` already carries everything needed to render range bars - `fromInstant` populates both
+`requestedTime` and `completionTime` from `HoodieInstant` (added under HUDI-9332) - so the UI could consume the v1
+timeline DTO directly. The v2 DTOs are not about exposing new fields; they are a deliberate, low-cost choice to give the
+new `/v2/` API a cleaner JSON contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the same source fields as v1, with UI-oriented JSON keys:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+
+ Versus v1, this renames `requestedTime`/`completionTime` to `requestTs`/`completionTs` and drops v1's redundant legacy
+ `ts` field (a duplicate of the requested time that the UI does not need).
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` - returns the table's `hoodie.properties` as a sorted JSON object.
+4. `getSchemaHistory(basePath, limit)` - reconstructs schema evolution from two sources; see
+ [Schema-History Reconstruction](#schema-history-reconstruction) below.
+
+Both `getTableConfig` and `getSchemaHistory` reuse a `HoodieTableMetaClient` cached per basepath in a
+`ConcurrentHashMap`, so repeated requests skip full metaClient construction. A `HoodieTableMetaClient` snapshots its
+`tableConfig` and `activeTimeline` at construction and refreshes them only on demand, so a build-once cache would serve
+stale config/schema for the lifetime of a long-lived embedded driver. To avoid that, each config/schema request first
+reloads the relevant snapshot under a per-basepath guard - `reloadTableConfig()` for `getTableConfig`,
+`reloadActiveTimeline()` for `getSchemaHistory` - before reading. These are human-click-frequency views, so the reload
+cost (one `hoodie.properties` read plus one timeline listing) is negligible and the response is always current. A TTL is
+deliberately not used: at this request rate it would save no meaningful work and would only reintroduce a staleness
+window. The UI also surfaces a **Refresh** control on the Table Config and Schema History tabs to re-pull on demand;
+because the server reloads on every request, the button simply re-issues the fetch rather than busting a cache. The
+timeline and instant-detail views are unaffected - they read the server-synced `FileSystemView`, not the cached
+metaClient.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Schema-History Reconstruction
+
+`getSchemaHistory` combines both schema sources Hudi maintains, so it returns something useful whether or not the table
+uses schema evolution:
+
+- **`currentSchema`** - the current table schema from `TableSchemaResolver.getTableSchema()`; `null` if it cannot be
+ resolved.
+- **`history`** - walks the completed commits timeline (`commit`/`deltacommit`/`replacecommit`), the most recent `limit`
+ instants only (default 200, capped at 1000), reading each instant's `HoodieCommitMetadata` and taking the schema under
+ `HoodieCommitMetadata.SCHEMA_KEY` (the `schema` entry in commit `extraMetadata`). An entry (`instant`,
+ `completionTime`, `action`, `schema`) is recorded only when the schema differs from the previous instant's, so runs of
+ commits carrying the same schema collapse to one change entry. Instants whose metadata cannot be read are skipped.
+- **`internalSchemaHistory`** (optional) - when `InternalSchema` is in use, the `.hoodie/.schema/` history string from
+ `FileBasedInternalSchemaStorageManager.getHistorySchemaStr()` is added for richer evolution tracking. It is omitted
+ when the `.schema` directory is absent - the common case for tables that never enabled schema evolution.
+
+**Cost model:** one schema resolve, at most `limit` completed-commit metadata reads (bounded further by the active
+timeline, since the archived timeline is out of scope), and one `.schema` history-file read. A table that never evolved
+its schema resolves `currentSchema`, collapses `history` to a single entry (or none if no commit recorded a schema), and
+omits `internalSchemaHistory` - no extra scanning and no error.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+10. User is able to click a **Refresh** control on the Table Config and Schema History tabs to re-pull the latest values
+ (the server reloads the table config / active timeline on each request)
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
+`http://<driver-host>:<embedded-server-port>/ui`. The embedded server port is logged at startup and can also be
+retrieved from the Spark configuration via the `hoodie.embed.timeline.server.port` property.
Review Comment:
Correct, fixed. Verified: `hoodie.embed.timeline.server.port` defaults to `0` ("picks a free port", HoodieWriteConfig.java:595) and the bound port is never written back to it - `EmbeddedTimelineService` keeps it in `serverPort` (line 180), logs it (181), and propagates it only via `FileSystemViewStorageConfig.withRemoteServerPort()` (210). Reading the property back would give `0`.
Reworded the sentence to drop the "retrieved from the Spark configuration" claim and point users at the startup log line (`Started embedded timeline server at <host>:<port>`) or the resolved `FileSystemViewStorageConfig` remote-server port instead.
(For the record, the `setValue(EMBEDDED_TIMELINE_SERVER_PORT_NUM, ...)` at HoodieWriteConfig.java:3512 is just the builder setter for a user-requested port, not a runtime write-back, so it doesn't change this.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3562775470
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
Review Comment:
The UI also bundles Bootstrap 5 and renderjson (see the QoL section and the `lib/bootstrap/`, `lib/renderjson/` assets), so vis-timeline is not the only bundled library and the ~300KB / vis-timeline-only figure in Dependency Impact undercounts the footprint. Update the File Structure tree and Dependency Impact to list all three vendored libraries, and note the LICENSE/NOTICE entries the ASF release requires for each.
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+Two v2 DTOs are introduced in a `v2` package to avoid modifying the existing `/v1/` API contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the v1 `InstantDTO` only exposes `action`, `timestamp`
Review Comment:
The v1 `InstantDTO` already exposes `completionTime` (and `requestedTime`) - `fromInstant` sets it from `HoodieInstant.getCompletionTime()`, added under HUDI-9332. The stated rationale that v1 lacks completion time no longer holds, so a new v2 `InstantDTO`/`TimelineDTOV2` layer is not required just to render range bars. If v2 is still wanted for the `requestTs`/`completionTs` JSON key naming or to drop the legacy `ts` field, state that as the reason; otherwise the UI can consume the existing v1 timeline DTO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315][RFC-94] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3401470018
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,515 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like
+`commit`, `deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer,
+but it's hard to visualize concurrent actions, spot missing transitions, or tell how long each step
+took. Debugging timeline issues by reading filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups
+related actions, and renders them in a time-ordered, interactive view. Users can track the lifecycle
+of each operation, see concurrency patterns, and spot anomalies or long-running tasks. The
+implementation extends `hudi-timeline-service` with new `/v2/` REST APIs and a static HTML +
+JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline), served via
+Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through
+metadata files. These files represent different actions (e.g., `deltacommit`, `compaction`) and
+their lifecycle states (`requested`, `inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and
+ day can be quickly determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you
+ manually calculate the difference between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck
+ compaction, which usually blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents
+requires custom Avro readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** -
+there is no support for starting or spawning jobs that mutate a Hudi table.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based
+ archive files is left for future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main
+ table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to
+ run in a trusted network, same as today.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as
+possible. Changes go into the existing `hudi-timeline-service` module, which contains a Javalin
+web-application that caches filesystem metadata of a Hudi table for job executors during
+tagging/writing.
+
+To use the Hudi Timeline UI, users can either start the Timeline Server in **STANDALONE** mode
+(which is already supported) or enable the UI on the **EMBEDDED** timeline server that runs within
+a Spark application's driver process (see [Configuration](#configuration)).
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab
+in the Spark UI links directly to the Hudi Timeline UI.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui/* - Static Files\n(HTML, JS, CSS)"]
+ API["/v2/timeline/* - UiHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode only"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** (`/ui/*`) - Javalin serves HTML, JavaScript, and CSS files from the
+ classpath (`src/main/resources/public/`). No server-side rendering or template engine is needed.
+2. **REST API requests** (`/v2/timeline/*`) - A new `UiHandler` processes these requests, reading
+ timeline data from the `FileSystemViewManager` and `HoodieTableMetaClient`, then returning JSON
+ responses.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's
+built-in static file serving handles files from the classpath - no template engine (e.g.,
+Thymeleaf) is needed and no new Java compile-time dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a
+text editor. The only external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled fallback copy of vis-timeline
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: CDN with Bundled Fallback
+
+The vis-timeline library is loaded with a two-tier strategy:
+
+1. **Primary:** Load from the `unpkg.com` CDN for automatic patch updates.
+2. **Fallback:** If the CDN is unreachable (e.g., air-gapped environments), load from the bundled
+ copy at `/lib/vis-timeline/`.
+
+The bundled fallback adds ~300KB to the JAR but ensures the UI works without internet access.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`,
+ `clustering`, `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These
+ correspond to the actions in `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to
+ `completionTime`. Non-completed instants (requested or inflight) are rendered as point items at
+ `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/timeline/instant/details` and
+ shows the deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths
+since the `FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|--------------------------------|----------------------------------------------------------------------|-------------------------|-------------------------------------------------------------------------------------|
+| GET | `/v2/timeline/instants` | `basepath` (required) | `List<InstantDTO>` (v2) | Returns all active instants with requested time, completion time, action, and state |
+| GET | `/v2/timeline/instant/details` | `basepath` (required), `instantTime` (required), `action` (required) | JSON string | Returns the deserialized content of a specific instant's metadata (Avro -> JSON) |
+
+Static files are served from the `/ui/` path, mapped to classpath resources under
+`src/main/resources/public/`.
+
+#### DTO Design
+
+A new v2 `InstantDTO` is introduced in a `v2` package to avoid modifying the existing `/v1/` API
+contract. The existing `InstantDTO` (in `o.a.h.common.table.timeline.dto`) only exposes `action`,
+`timestamp` (requested time), and `state` - it lacks `completionTime`, which is needed for
+rendering range bars on the timeline.
+
+The v2 `InstantDTO` contains:
Review Comment:
Good catch, the current wording reads as per-request but that is not the intent. `UiHandler` caches one `HoodieTableMetaClient` per basepath in a `ConcurrentHashMap`, built once on first access and reused for every detail click. This mirrors the per-basepath caching that `InstantStateHandler` and `MarkerHandler` already use, and matches how `FileSystemViewManager` builds a single metaClient per basepath. So clicking through instants only pays the targeted Avro read, not metaClient construction. Updated the Handler Design section to make this explicit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(ui): [HUDI-9315] Add RFC-94: Hudi Timeline UI [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #13152:
URL: https://github.com/apache/hudi/pull/13152#discussion_r3408537004
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+Two v2 DTOs are introduced in a `v2` package to avoid modifying the existing `/v1/` API contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the v1 `InstantDTO` only exposes `action`, `timestamp`
+ (requested time), and `state`; it lacks completion time, needed for rendering range bars. The v2 `InstantDTO` has:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
Review Comment:
🤖 The cached `HoodieTableMetaClient` is built once per basepath, but there's no invalidation strategy described. In an embedded server running inside a long-lived Spark driver, the table's schema or `hoodie.properties` can change after the cache is populated — the UI will then show stale values for the rest of the driver's lifetime. Could you spell out a refresh approach (TTL, reload on each request, or an explicit invalidation hook tied to timeline changes)?
<sub><i>- AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
+optional `from`/`to` time-range query params (and/or a `limit`) without breaking the existing contract.
+
+#### DTO Design
+
+Two v2 DTOs are introduced in a `v2` package to avoid modifying the existing `/v1/` API contract:
+
+- **`InstantDTO`** (`o.a.h.common.table.timeline.dto.v2`) - the v1 `InstantDTO` only exposes `action`, `timestamp`
+ (requested time), and `state`; it lacks completion time, needed for rendering range bars. The v2 `InstantDTO` has:
+ - `action` - the action type (e.g., `commit`, `deltacommit`, `compaction`)
+ - `requestedTime` (JSON `requestTs`) - requested timestamp (`HoodieInstant.requestedTime()`)
+ - `completionTime` (JSON `completionTs`) - completion timestamp (`HoodieInstant.getCompletionTime()`), null for
+ non-completed instants
+ - `state` - the instant state (`REQUESTED`, `INFLIGHT`, `COMPLETED`)
+- **`TimelineDTOV2`** - wraps a `List<InstantDTO>` (`instants`); this is what `/v2/hoodie/view/timeline/instants/all`
+ returns.
+
+#### Handler Design
+
+The v2 endpoints are served by the existing `TimelineHandler` (which already serves the v1 timeline routes); a separate
+`UiHandler` serves only the UI entry page.
+
+`TimelineHandler` methods:
+
+1. `getTimelineV2(basePath)` - maps `viewManager.getFileSystemView(basePath).getTimeline()` to a `TimelineDTOV2` (the
+ active timeline's instants, each including completion time).
+2. `getInstantDetails(basePath, instant, action, state)` - reads the instant's Avro content via the active timeline's
+ `getInstantDetails()` and deserializes it to JSON. The instant is created with the timeline's own layout-aware
+ `InstantGenerator`; a malformed `state`/`action` returns 400, a read failure is logged and returns 500.
+3. `getTableConfig(basePath)` / `getSchemaHistory(basePath, limit)` - serve the table-config and schema-history views.
+ Both reuse a `HoodieTableMetaClient` cached per basepath in a `ConcurrentHashMap` (built once on first access), so
+ repeated requests pay only the targeted read, not metaClient construction.
+
+`UiHandler` registers `GET /ui`, returning `/public/index.html` from the classpath as the UI entry page.
+
+#### Registration in RequestHandler
+
+The v2 routes are registered following the existing pattern:
+
+- The v1 timeline routes remain registered unconditionally in `registerTimelineAPI()`.
+- The v2 UI routes are registered in `registerTimelineV2API()`, called from `register()` only when `--enable-ui` is set.
+ `UiHandler` (serving `/ui`) and the static-file serving are gated by the same flag.
+
+#### Error Handling
+
+- **Invalid basepath** -> HTTP 400 with a descriptive error message (e.g., "Not a valid Hudi table path").
+- **Empty timeline** -> Returns an empty list `[]`. The frontend displays "No instants found".
+- **Failed instant detail read** -> HTTP 500 with error details (e.g., Avro deserialization failure).
+
+### Feature
+
+The first cut presents three read-only tabs for a Hudi table: **Timeline**, **Table Config**, and **Schema History**.
+
+The permitted user actions are:
+
+1. User is able to input a Hudi table basepath
+2. User is able to click submit after inputting Hudi table basepath
+3. The timeline of the Hudi table is rendered
+4. User is able to scroll through timeline (horizontally)
+5. User is able to zoom in and out of timeline
+6. User is able to hover over instant for more details
+7. User is able to click on a specific instant and the JSON string of the timeline details are rendered
+8. User is able to view the table's configuration (`hoodie.properties`) in the Table Config tab
+9. User is able to view the table's schema and schema-change history in the Schema History tab
+
+Each action type occupies its own horizontal row so concurrent actions are visually separated. Completed instants appear
+as horizontal bars whose width represents duration (requested -> completed). Inflight and requested instants appear as
+point markers. Color indicates state: green for completed, yellow for inflight, red for requested.
+
+### Examples
+
+Proof of concept (PoC) snapshots:
+
+**Main Page with Timeline Rendered**
+
+
+**Hovering Over an Instant**
+
+
+**Selecting an Instant**
+
+
+## Configuration
+
+### Standalone Mode
+
+To start the Timeline Server in standalone mode with the UI enabled:
+
+```shell
+java -cp hudi-timeline-server-bundle-*.jar \
+ org.apache.hudi.timeline.service.TimelineService \
+ --server-port 26754 \
+ --enable-ui
+```
+
+Once started, the UI is accessible at `http://localhost:26754/ui`.
+
+The server port is configurable via the existing `--server-port` (or `-p`) flag (default: `26754`). The `--enable-ui`
+flag controls whether the UI static files, the `/ui` page, and the `/v2/hoodie/view/` UI API endpoints are registered.
+When the flag is not set, the timeline server behaves exactly as it does today - no UI-related routes are added.
+
+### Embedded Mode (Spark-Shell / Spark Driver)
+
+> **Status: deferred to a follow-up.** Embedded-mode UI enablement is intentionally split out of the initial PR to keep
+> it small: the standalone UI ships first, then the embedded server is wired to enable it. The design below is retained
+> for that follow-up.
+
+When running Hudi inside a Spark application, the `EmbeddedTimelineService` already starts a timeline server within the
+driver process. The UI can be enabled on this embedded server by setting a Spark configuration property:
+
+```
+hoodie.embed.timeline.server.ui.enable = true
+```
+
+This property defaults to `false`. When set to `true`, the embedded timeline server registers the same UI routes and
+static file serving as the standalone mode.
+
+#### Starting from spark-shell
+
+```shell
+spark-shell \
+ --packages org.apache.hudi:hudi-spark3-bundle_2.12:1.2.0 \
+ --conf "hoodie.embed.timeline.server.ui.enable=true"
+```
+
+Once a write operation initializes the `EmbeddedTimelineService`, the UI becomes available at
Review Comment:
🤖 The Module Placement note says `EmbeddedTimelineService` (in `hudi-client-common`) coordinates registration by calling into the Spark-specific tab class in `hudi-spark-client`, but `hudi-client-common` cannot depend on `hudi-spark-client`. Could you specify the dispatch mechanism — `ServiceLoader`/SPI, reflection on a fully-qualified class name, or a callback registered through `HoodieEngineContext`? This is the kind of cross-module wiring that's worth pinning down before implementation. @bvaradar this touches a module-boundary contract worth a quick sanity check.
<sub><i>- AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
Review Comment:
🤖 Could you specify how the schema-change history is reconstructed? The endpoint says "schema-change history from recent commits" with a default `limit=200` (max `1000`), but the actual source is ambiguous — scanning commit `extraMetadata` for `schema` keys, reading `.hoodie/.schema/` when `InternalSchema` is on, or both? The cost model (per-commit metadata read for up to 1000 commits) and the behavior for tables that never use schema evolution depend on the answer.
<sub><i>- AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
+a stable entry URL.
+
+**On response size and pagination:** `GET /v2/hoodie/view/timeline/instants/all` returns the full active timeline. The
+active timeline is bounded by archiving (the unbounded archived timeline is out of scope), so instant counts are
+typically modest. The first cut intentionally returns all active instants and relies on client-side zoom/scroll and
+filtering for navigation. If active-timeline sizes become a concern, the endpoint can be extended additively with
Review Comment:
🤖 Mounting the static assets at the server root (`/js/...`, `/css/...`, `/lib/...`) reserves these path prefixes from any future `/v1/`, `/v2/`, or other module-registered routes on the same Javalin instance. Could you namespace them under `/ui/static/...` (matching the `/ui` entry page) so the UI surface and the API surface don't collide as new endpoints are added?
<sub><i>- AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
##########
rfc/rfc-94/rfc-94.md:
##########
@@ -0,0 +1,545 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# RFC-94: Hudi Timeline User Interface (UI)
+
+## Proposers
+
+- @voonhous
+
+## Approvers
+
+- @danny0405
+- @rahil-c
+- @yihua
+
+## Status
+
+JIRA: [HUDI-9315](https://issues.apache.org/jira/browse/HUDI-9315)
+
+## Abstract
+
+Hudi Timeline metadata is stored as timestamped files representing state transitions of actions like `commit`,
+`deltacommit` and `compaction`. These files are accessible via the CLI or a file explorer, but it's hard to visualize
+concurrent actions, spot missing transitions, or tell how long each step took. Debugging timeline issues by reading
+filenames is tedious.
+
+This RFC proposes a UI-based timeline visualization tool that parses these metadata files, groups related actions, and
+renders them in a time-ordered, interactive view. Users can track the lifecycle of each operation, see concurrency
+patterns, and spot anomalies or long-running tasks. The implementation extends `hudi-timeline-service` with new `/v2/`
+REST APIs and a static HTML + JavaScript frontend powered by [vis-timeline](https://github.com/visjs/vis-timeline),
+served via Javalin's built-in static file serving with zero new Java compile-time dependencies.
+
+## Background
+
+Today, we rely on the CLI or direct filesystem inspection to understand timeline state through metadata files. These
+files represent different actions (e.g., `deltacommit`, `compaction`) and their lifecycle states (`requested`,
+`inflight`, `completed`), encoded in file names like:
+
+```shell
+20250409102118815.deltacommit.inflight
+20250409102118815.deltacommit.requested
+20250409102118815_20250409102124339.deltacommit
+20250409102121593.compaction.inflight
+20250409102121593.compaction.requested
+20250409102121593_20250409102122232.commit
+20250409102124581.deltacommit.inflight
+20250409102124581.deltacommit.requested
+20250409102124581_20250409102125667.deltacommit
+20250409102124612.compaction.inflight
+20250409102124612.compaction.requested
+20250409102124612_20250409102124892.commit
+20250409102127348.deltacommit.inflight
+20250409102127348.deltacommit.requested
+20250409102127348_20250409102128481.deltacommit
+20250409102127500.compaction.inflight
+20250409102127500.compaction.requested
+20250409102127500_20250409102127721.commit
+```
+
+This works, but has a few problems:
+
+1. No visibility into concurrency
+ - Multiple actions (e.g., `deltacommit` and `compaction`) often run concurrently.
+ - The CLI doesn't help correlate or visualize overlapping operations.
+2. Lack of temporal context
+ - Timestamps are embedded in filenames but are hard to compare visually - year, month and day can be quickly
+ determined, but minutes and seconds are harder to parse.
+ - No easy way to tell how long an action took or whether it's stalling unless you manually calculate the difference
+ between requested and completion time.
+3. Hard to spot inconsistencies or missing states
+ - An `inflight` compaction without a corresponding `commit` can indicate a starved/stuck compaction, which usually
+ blocks archiving/cleaning.
+ - These gaps are easy to miss when scanning filenames.
+
+On top of that, all timeline files are now stored as Avro binaries. Inspecting their contents requires custom Avro
+readers to convert the binaries to JSON.
+
+## Scope
+
+This RFC covers visualization of metadata available in Hudi tables. All features are **READ-ONLY** - there is no support
+for starting or spawning jobs that mutate a Hudi table.
+
+Alongside the timeline, the UI surfaces two additional read-only metadata views: the table's configuration
+(`hoodie.properties`) and its schema-change history.
+
+The following are **out of scope**:
+
+- **Archived timeline:** Only the active timeline is rendered. Loading instants from LSM-based archive files is left for
+ future work.
+- **Metadata table overlay:** The metadata table's own timeline is not shown alongside the main table timeline.
+- **Write/mutation operations:** The UI cannot trigger compactions, clustering, or any write action.
+- **Authentication/authorization:** No access control is added. The timeline server is assumed to run in a trusted
+ network, same as today.
+
+ **Threat model:** The UI does not widen the timeline server's exposure surface. The `/v2/` endpoints read the same
+ active-timeline and filesystem metadata that the existing `/v1/` REST APIs already serve, on the same network
+ interface (the server binds to all interfaces on the driver/standalone host). The UI is also opt-in and off by default
+ (`--enable-ui`). Operators on untrusted networks should front the server with a reverse proxy or restrict it to a
+ private interface / localhost via network policy.
+
+## Implementation
+
+Keeping the implementation lightweight is a priority - we should add as few dependencies as possible. Changes go into
+the existing `hudi-timeline-service` module, which contains a Javalin web-application that caches filesystem metadata of
+a Hudi table for job executors during tagging/writing.
+
+The first cut runs the UI on the Timeline Server in **STANDALONE** mode (see [Configuration](#configuration)) and is
+self-contained within `hudi-timeline-service`. Enabling the UI on the **EMBEDDED** timeline server inside a Spark
+driver, together with a Spark UI tab, requires cross-module wiring (`hudi-client-common`, `hudi-spark-client`); it is
+designed below but deferred to a follow-up to keep the initial PR small and focused. The standalone UI lands first; the
+embedded/Spark linking lands next.
+
+The Hudi Timeline UI has two parts: the frontend and backend.
+
+### Architecture
+
+The timeline server can run standalone or embedded inside a Spark driver. In embedded mode, a tab in the Spark UI links
+directly to the Hudi Timeline UI. The embedded mode and Spark UI tab (right side of the diagram below) are a planned
+follow-up; the first cut is standalone-only.
+
+```mermaid
+graph LR
+ Browser["Browser"]
+
+ subgraph Driver["Standalone / Spark Driver"]
+ subgraph TimelineServer["Javalin (Timeline Server)"]
+ Static["/ui + assets at root\n(HTML, JS, CSS)"]
+ API["/v2/hoodie/view/* - TimelineHandler"]
+ FSVM["FileSystemViewManager"]
+ Meta["HoodieTimeline / MetaClient"]
+
+ API --> FSVM --> Meta
+ end
+
+ subgraph SparkUI["Spark UI (:4040) - embedded mode (follow-up)"]
+ direction TB
+ SparkUIPad[ ] ~~~ Tabs["[Jobs] [Stages] ... [Hudi Timeline]"]
+ end
+
+ style SparkUIPad fill:none,stroke:none,color:none
+
+ Tabs -- "link" --> Static
+ end
+
+ Browser -- "HTTP" --> Static
+ Browser -- "HTTP" --> API
+ Browser -. "HTTP\n(embedded mode)" .-> SparkUI
+```
+
+There are two categories of requests:
+
+1. **Static file requests** - Javalin serves HTML, JavaScript, and CSS files from the classpath
+ (`src/main/resources/public/`) at the server root; `UiHandler` serves `index.html` at `/ui`. No server-side
+ rendering or template engine is needed.
+2. **REST API requests** (`/v2/hoodie/view/*`) - `TimelineHandler` processes these requests, reading timeline data from
+ the `FileSystemViewManager` (and a per-basepath `HoodieTableMetaClient` for table config/schema), returning JSON.
+
+### Frontend
+
+The frontend is static HTML pages with vanilla JavaScript, similar to the Spark Web UI. Javalin's built-in static file
+serving handles files from the classpath - no template engine (e.g., Thymeleaf) is needed and no new Java compile-time
+dependencies are added.
+
+No frontend build pipeline (npm, webpack, vite) is needed. Contributing to the UI requires only a text editor. The only
+external library is vis-timeline for timeline rendering.
+
+#### File Structure
+
+```
+hudi-timeline-service/src/main/resources/public/
+├── index.html # Landing page with basepath input form
+├── js/
+│ └── timeline.js # vis-timeline initialization and REST API calls
+├── css/
+│ └── style.css # Basic styling
+└── lib/
+ └── vis-timeline/ # Bundled vis-timeline assets
+ ├── vis-timeline-graph2d.min.js
+ └── vis-timeline-graph2d.min.css
+```
+
+#### JavaScript Delivery: Bundled, No External Calls
+
+The vis-timeline library is served from the bundled copy at `/lib/vis-timeline/`. The UI makes no external network
+calls, so it works out of the box in air-gapped and security-conscious deployments with no extra configuration. The
+bundled assets add ~300KB to the JAR.
+
+Pinning a vendored copy (rather than loading from a CDN) keeps the UI deterministic and avoids a runtime dependency on
+an external host being reachable. If automatic patch updates are wanted later, a CDN source can be added as an opt-in
+config flag without changing this default.
+
+#### vis-timeline Configuration
+
+The timeline is configured with groups and items that map to Hudi's timeline model:
+
+- **Groups:** One row per action type - `commit`, `deltacommit`, `compaction`, `clean`, `rollback`, `clustering`,
+ `savepoint`, `logcompaction`, `indexing`, `restore`, `replacecommit`. These correspond to the actions in
+ `HoodieTimeline.VALID_ACTIONS_IN_TIMELINE`.
+- **Items:** Completed instants are rendered as range bars spanning from `requestedTime` to `completionTime`.
+ Non-completed instants (requested or inflight) are rendered as point items at `requestedTime`.
+- **Color coding:** Items are colored by state:
+ - Green -> `COMPLETED`
+ - Yellow -> `INFLIGHT`
+ - Red -> `REQUESTED`
+- **Tooltip:** On hover, shows the action type, requested time, completion time, and duration.
+- **Click handler:** Clicking an instant fetches its detail via `/v2/hoodie/view/timeline/instant` and shows the
+ deserialized JSON in a detail panel below the timeline.
+
+### Backend
+
+A `hudi-timeline-service` instance already serves filesystem metadata for multiple table basePaths since the
+`FileSystemView`s are cached in a map keyed by basepath.
+
+We extend this module with `/v2/` APIs to serve the timeline metadata needed by the UI.
+
+#### API Specification
+
+| Method | Path | Parameters | Response | Description |
+|--------|-----------------------------------------|-----------------------------------------------------------------------|-----------------|----------------------------------------------------------------------------------------------|
+| GET | `/v2/hoodie/view/timeline/instants/all` | `basepath` (required) | `TimelineDTOV2` | All active instants (each with requested time, completion time, action, state), wrapped in a timeline DTO |
+| GET | `/v2/hoodie/view/timeline/instant` | `basepath`, `instant`, `instantaction`, `instantstate` (all required) | JSON string | Deserialized content of a specific instant's metadata (Avro -> JSON) |
+| GET | `/v2/hoodie/view/table/config` | `basepath` (required) | JSON object | The table's `hoodie.properties` (sorted) |
+| GET | `/v2/hoodie/view/table/schema/history` | `basepath` (required), `limit` (optional, default 200, max 1000) | JSON object | Current table schema plus schema-change history from recent commits |
+
+Static files (HTML, JS, CSS) are served from the classpath under `src/main/resources/public/` at the server root (e.g.,
+`/js/timeline.js`, `/lib/...`). `UiHandler` additionally registers `GET /ui`, which returns `index.html` to give the UI
Review Comment:
🤖 The `/v2/hoodie/view/table/config` endpoint returns the full `hoodie.properties` content. These can include sensitive material — encryption key references, lock-provider connection strings, KMS endpoints, vault paths, etc. The threat-model note argues parity with `/v1/`, but `/v1/` does not expose `hoodie.properties`. Could you either filter known-sensitive keys (allowlist or denylist) or explicitly call out this expanded exposure in the threat model so operators understand what's on the wire?
<sub><i>- AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org