You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "rangareddy (via GitHub)" <gi...@apache.org> on 2026/08/31 04:18:18 UTC
[PR] docs(cli): document the 20 undocumented CLI commands and refresh the help listing [hudi]
rangareddy opened a new pull request, #19799:
URL: https://github.com/apache/hudi/pull/19799
### Describe the issue this Pull Request addresses
Closes part of #15376 (JIRA: [HUDI-4689](https://issues.apache.org/jira/browse/HUDI-4689), epic
[HUDI-1388](https://issues.apache.org/jira/browse/HUDI-1388)), which asks for documentation covering all CLI
commands.
`hudi-cli` on `master` exposes **106 commands across 26 command classes**. `website/docs/cli.md` covered 86 of
them. The 20 that were missing entirely:
| Command class | Undocumented commands |
| --- | --- |
| `LockAuditingCommand` | `locks audit enable` / `disable` / `status` / `validate` / `cleanup` |
| `TimelineCommand` | `timeline show active` / `show incomplete`, `metadata timeline show active` / `show incomplete` |
| `RepairsCommand` | `rename partition`, `repair deprecated partition`, `repair show empty commit metadata` |
| `DiffCommand` | `diff file`, `diff partition` |
| `RestoresCommand` | `show restore`, `show restores` |
| `ArchivedCommitsCommand` | `trigger archival` |
| `KerberosAuthenticationCommand` | `kerberos kdestroy` |
| `MetadataCommand` | `metadata delete-record-index` |
| `TableCommand` | `table set-meta-fields-mode` |
The `help` output embedded in the page had drifted too, so a reader could not even discover those commands
existed from the docs.
### Summary and Changelog
One file, `website/docs/cli.md`.
- **The `help` listing now carries all 106 commands.** Entries were inserted at their alphabetical positions
rather than the block being regenerated, so the diff stays reviewable.
- **Five new reference sections**, each giving the command's purpose, its options and their defaults:
- `Auditing Storage Locks` — the five `locks audit` commands, the on-disk layout
(`<basePath>/.hoodie/.locks/audit_enabled.json` for the config, `<basePath>/.hoodie/.locks/audit/` for the
JSONL records), and what `validate` treats as an error (overlapping transaction windows) versus a warning
(a transaction that never released its lock).
- `Inspecting the Timeline` — the four timeline commands, their output columns, and which options apply to
which.
- `Diffing a File or Partition` — `diff file` and `diff partition`.
- `Repairing a Table` — `rename partition`, `repair deprecated partition`,
`repair show empty commit metadata`.
- `Changing the Meta Fields Mode` — `table set-meta-fields-mode` and its two guards.
- **Four more folded into the sections they belong to**: `trigger archival` under Archived Commits,
`show restore` / `show restores` under Savepoint and Restore, `kerberos kdestroy` under the Kerberos
section, `metadata delete-record-index` under Record Index Lookup.
Every option name, default and behaviour was read from the command classes on `master`, not from the help
text alone. That matters, because in three places the two disagree or the help text is silent:
- **`diff file` and `diff partition` ignore `--startTs` and `--endTs` unless `--includeArchivedTimeline` is
also `true`.** `CLIUtils#getTimelineInRange` (`:46-62`) applies the bounds only on the archived branch and
returns the whole active timeline otherwise, so a narrow range silently does nothing. The help text
advertises the defaults with no hint of the precondition. Documented explicitly.
- **`repair show empty commit metadata` returns `void`** and reports through `log.warn`, so it renders no
result table; the findings land in the CLI log. Documented, rather than implying it prints a result.
- **`rename partition` physically removes the old partition directory** after its `delete_partition` write
(`SparkMain#renamePartition`), while `repair deprecated partition` leaves the emptied `default` directory in
place. The two are otherwise the same shape, so the difference is easy to miss.
### Verification
```
cd website && npm run build
-> [SUCCESS] Generated static files in "build"
docs markdown link-format checks (the ones asf-site.ci.yml runs on website/docs/**)
-> no absolute /docs/ paths, no links missing .md, no ../ or ./ prefixes
coverage recheck, parsing @ShellMethod keys out of hudi-cli on master and grepping cli.md
-> 106 source commands, 0 missing from cli.md
-> 20/20 target commands documented beyond the help listing
```
The five new sections were confirmed present in the rendered `build/docs/next/cli/index.html`.
### Impact
Docs only; no code or config changes. The page is the current (`next`) version, so it tracks `master`, which
is where these commands live. Versioned copies under `website/versioned_docs/` are deliberately untouched,
since several of these commands do not exist in those releases.
**This does not close HUDI-4689.** 56 of the 106 commands are still listing-only entries with no prose
section. All of them predate this change; this PR closes the gap for the 20 that had no documentation at all.
Happy to keep chipping at the remainder in follow-ups if that is the preferred shape, or to fold more in here.
### Risk Level
none
### Documentation Update
This is the documentation update.
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
- [x] CI passes on my PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(cli): document all 106 CLI commands [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #19799:
URL: https://github.com/apache/hudi/pull/19799#discussion_r3916380785
##########
website/docs/cli.md:
##########
@@ -698,6 +769,23 @@ hudi:trips->metadata lookup-record-index --record_key <key> --partition_path <pa
The `--partition_path` argument is optional for a global RLI (where record keys are unique across all partitions) and required for a partitioned RLI. If `--partition_path` is omitted for a partitioned RLI, the command will return an error. The output columns are `Record key`, `Partition path`, `File Id`, and `Instant time`.
+To drop the record index partition from the Metadata Table entirely, for example before rebuilding it:
+
+```shell
+hudi:trips->metadata delete-record-index --backup true
Review Comment:
**major:** This transcript cannot be produced from a fresh session: `deleteRecordIndex` passes `new HoodieSparkEngineContext(jsc)` (`MetadataCommand.java:156`) and `jsc` is only set by `metadata create` / `init` / `list-partitions` (`:460`), so `connect` followed by this command NPEs in `jsc.sc()`. `metadata delete` (`:143`) uses `HoodieLocalEngineContext`, and `deleteMetadataTablePartition` never reads the context. Could we note that prerequisite here, and follow up with a one-line master fix to `HoodieLocalEngineContext` so the output shown is reachable?
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (empty by default) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
Review Comment:
**major:** The site's default docs version is 1.2.0, not `next` (`versions.json`; `docusaurus.config.js:563-565` maps `1.2.0` to path `""`), so `hudi.apache.org/docs/cli` will not show any of this. On `release-1.2.0` every command documented here exists with the same options except `table set-meta-fields-mode`, and the older spelling there is `commits show_infights`. Could we mirror the change into `versioned_docs/version-1.2.0/cli.md`, minus the Meta Fields Mode section and reference entry?
##########
website/docs/cli.md:
##########
@@ -253,29 +269,43 @@ hudi:trips->help
* create - Create a hoodie table if not present
* date - Displays the local date and time
* desc - Describe Hoodie Table properties
+* diff file - Check how file differs across range of commits
+* diff partition - Check how file differs across range of commits. It is meant to be used only for partitioned tables.
* downgrade table - Downgrades a table
* exit - Exits the shell
* export instants - Export Instants and their metadata from the Timeline
* fetch table schema - Fetches latest table schema
* hdfsparquetimport - Imports Parquet table to a hoodie table
Review Comment:
**major:** `hdfsparquetimport` no longer exists: `HDFSParquetImportCommand` was deleted in aeebfcfcec27 (#12695), and the new Command reference correctly omits it, so the listing now contradicts the reference. Could we drop the line while this block is being reconciled?
```suggestion
```
##########
website/docs/cli.md:
##########
@@ -253,29 +269,43 @@ hudi:trips->help
* create - Create a hoodie table if not present
* date - Displays the local date and time
* desc - Describe Hoodie Table properties
+* diff file - Check how file differs across range of commits
+* diff partition - Check how file differs across range of commits. It is meant to be used only for partitioned tables.
* downgrade table - Downgrades a table
* exit - Exits the shell
* export instants - Export Instants and their metadata from the Timeline
* fetch table schema - Fetches latest table schema
* hdfsparquetimport - Imports Parquet table to a hoodie table
* help - List all commands usage
+* kerberos kdestroy - Destroy Kerberos authentication
Review Comment:
**major:** The listing still misses four commands that exist on master, so it covers 102 rather than all 106: `commits show_inflights` (after line 254), `kerberos kinit` (here, next to its new counterpart), `metadata lookup-record-index` (after line 292) and `table change-table-type` (before line 332). Could we add them, starting with:
```suggestion
* kerberos kdestroy - Destroy Kerberos authentication
* kerberos kinit - Perform Kerberos authentication
```
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (empty by default) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
+
+Every command `hudi-cli` exposes, grouped by area, with its options and their defaults. An option marked
+`(required)` has no default and must be supplied; a value in backticks after an option is its default.
+The sections above cover the commonly used ones in more depth.
+
+Some entries are aliases of the same command rather than distinct ones. `refresh`, `metadata refresh`,
+`commits refresh`, `cleans refresh` and `savepoints refresh` are five names for one method that reloads the table
+metadata, and `temp query` / `temp_query`, `temp delete` / `temp_delete` and `temps show` / `temps_show` are
+underscore and space spellings of the same three commands.
+
+### Table and session
+
+- **`cleans refresh`** Refresh table metadata.
+- **`commits refresh`** Refresh table metadata.
+- **`connect`** Connect to a hoodie table.
+ <br />Options: `--path` (required), `--eventuallyConsistent` (`false`), `--initialCheckIntervalMs` (`2000`), `--maxWaitIntervalMs` (`300000`), `--maxCheckIntervalMs` (`7`), `--timeGeneratorType` (`WAIT_TO_ADJUST_SKEW`), `--maxExpectedClockSkewMs` (`200`), `--useDefaultLockProvider` (`false`)
+- **`create`** Create a hoodie table if not present.
+ <br />Options: `--path` (required), `--tableName` (required), `--tableType` (`COPY_ON_WRITE`), `--archiveLogFolder`, `--tableVersion`, `--payloadClass` (`org.apache.hudi.common.model.HoodieAvroPayload`)
+- **`desc`** Describe Hoodie Table properties.
+- **`fetch table schema`** Fetches latest table schema.
+ <br />Options: `--outputFilePath`
+- **`kerberos kdestroy`** Destroy Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`)
+- **`kerberos kinit`** Perform Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`), `--principal` (required), `--keytab` (required)
+- **`metadata refresh`** Refresh table metadata.
+- **`refresh`** Refresh table metadata.
+- **`savepoints refresh`** Refresh table metadata.
+- **`set`** Set spark launcher env to cli.
+ <br />Options: `--conf` (required)
+- **`show env`** Show spark launcher env by key.
+ <br />Options: `--key` (required)
+- **`show envs all`** Show spark launcher envs.
+- **`table change-table-type`** Change hudi table type to target type: COW or MOR.
+ <br />Options: `--target-type` (required), `--enable-compaction` (`true`), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`table delete-configs`** Delete the supplied table configs from the table.
+ <br />Options: `--comma-separated-configs` (required)
+- **`table recover-configs`** Recover table configs, from update/delete that failed midway.
+- **`table set-meta-fields-mode`** Set hoodie.meta.fields.mode on an existing table. This is the sanctioned way to change.
+ <br />Options: `--target-mode` (required), `--force` (`false`)
+- **`table update-configs`** Update the table configs with configs with provided file.
+ <br />Options: `--props-file` (required)
+- **`utils loadClass`** Load a class.
+ <br />Options: `--class` (required)
+
+### Commits and the timeline
+
+- **`commit show_write_stats`** Show write stats of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showfiles`** Show file level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showpartitions`** Show partition level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commits compare`** Compare commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`commits show`** Show the commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`, `--includeArchivedTimeline` (`false`)
+- **`commits show_inflights`** Show inflight instants that are left longer than a certain duration.
+ <br />Options: `--lookbackInMins` (`0`)
+- **`commits showarchived`** Show the archived commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`commits sync`** Sync commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`diff file`** Check how file differs across range of commits.
+ <br />Options: `--fileId` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`diff partition`** Check how file differs across range of commits. It is meant to be used only for partitioned tables.
+ <br />Options: `--partitionPath` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`metadata timeline show active`** List all instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`metadata timeline show incomplete`** List all incomplete instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`show archived commit stats`** Read commits from archived files and show file group details.
+ <br />Options: `--archiveFolderPattern`, `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show archived commits`** Read commits from archived files and show details.
+ <br />Options: `--skipMetadata` (`true`), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`timeline show active`** List all instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--with-metadata-table` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`timeline show incomplete`** List all incomplete instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`trigger archival`** Trigger archival.
+ <br />Options: `--minCommits` (`20`), `--maxCommits` (`30`), `--commitsRetainedByCleaner` (`10`), `--enableMetadata` (`true`), `--sparkMemory` (`1G`), `--sparkMaster` (`local`)
+
+### Files, stats and log files
+
+- **`show fsview all`** Show entire file-system view.
+ <br />Options: `--pathRegex` (`*`), `--baseFileOnly` (`false`), `--maxInstant`, `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show fsview latest`** Show latest file-system view.
+ <br />Options: `--partitionPath`, `--baseFileOnly` (`false`), `--maxInstant`, `--merge` (`true`), `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile metadata`** Read commit metadata from log files.
+ <br />Options: `--logFilePathPattern` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile records`** Read records from log files.
+ <br />Options: `--limit` (`10`), `--logFilePathPattern` (required), `--mergeRecords` (`false`)
+- **`stats filesizes`** File Sizes. Display summary stats on sizes of files.
+ <br />Options: `--partitionPath` (`*/*/*`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`stats wa`** Write Amplification. Ratio of how many records were upserted to how many.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+
+### Table services
+
+- **`clean showpartitions`** Show partition level details of a clean.
+ <br />Options: `--clean` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`cleans run`** Run clean.
+ <br />Options: `--sparkMemory` (`4G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster`
+- **`cleans show`** Show the cleans.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--startTs`, `--endTs`, `--includeArchivedTimeline` (`false`), `--desc` (`false`), `--headeronly` (`false`)
+- **`clustering run`** Run Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--clusteringInstant`, `--propsFilePath`, `--hoodieConfigs`
+- **`clustering schedule`** Schedule Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`1g`), `--propsFilePath`, `--hoodieConfigs`
+- **`clustering scheduleAndExecute`** Run Clustering. Make a cluster plan first and execute that plan immediately.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`compaction repair`** Renames the files to make them consistent with the timeline as.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction run`** Run Compaction for given instant time.
+ <br />Options: `--parallelism` (`3`), `--schemaFilePath`, `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--compactionInstant`, `--propsFilePath`, `--hoodieConfigs`
+- **`compaction schedule`** Schedule Compaction.
+ <br />Options: `--sparkMemory` (`1G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster` (`local`)
+- **`compaction scheduleAndExecute`** Schedule compaction plan and execute this plan.
+ <br />Options: `--parallelism` (`3`), `--schemaFilePath`, `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`compaction show`** Shows compaction details for a specific compaction instant.
+ <br />Options: `--instant` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`compaction showarchived`** Shows compaction details for a specific compaction instant.
+ <br />Options: `--instant` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`compaction unschedule`** Unschedule Compaction.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--skipValidation` (`false`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction unscheduleFileId`** UnSchedule Compaction for a fileId.
+ <br />Options: `--fileId` (required), `--partitionPath`, `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--skipValidation` (`false`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction validate`** Validate Compaction.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compactions show all`** Shows all compactions that are in active timeline.
+ <br />Options: `--includeExtraMetadata` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compactions showarchived`** Shows compaction details for specified time window.
+ <br />Options: `--includeExtraMetadata` (`false`), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`marker delete`** Delete the marker.
+ <br />Options: `--commit` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`1G`)
+
+### Rollback, savepoint, restore and repair
+
+- **`commit rollback`** Rollback a commit.
+ <br />Options: `--commit` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`), `--rollbackUsingMarkers` (`false`)
+- **`downgrade table`** Downgrades a table.
+ <br />Options: `--toVersion`, `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`rename partition`** Rename partition. Usage: rename partition --oldPartition <oldPartition> --newPartition <newPartition>.
+ <br />Options: `--oldPartition` (required), `--newPartition` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`repair addpartitionmeta`** Add partition metadata to a table, if not present.
+ <br />Options: `--dryrun` (`true`)
+- **`repair corrupted clean files`** Repair corrupted clean files.
+- **`repair deduplicate`** De-duplicate a partition path contains duplicates & produce repaired files to replace with.
+ <br />Options: `--duplicatedPartitionPath`, `--repairedOutputPath` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`), `--dryrun` (`true`), `--dedupeType` (`insert_type`)
+- **`repair deprecated partition`** Repair deprecated partition ("default"). Re-writes data from the deprecated partition into.
+ <br />Options: `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`repair migrate-partition-meta`** Migrate all partition meta file currently stored in text format.
+ <br />Options: `--dryrun` (`true`)
+- **`repair overwrite-hoodie-props`** Overwrite hoodie.properties with provided file. Risky operation. Proceed with caution!.
+ <br />Options: `--new-props-file` (required)
+- **`repair show empty commit metadata`** Show failed commits.
+- **`savepoint create`** Savepoint a commit.
+ <br />Options: `--commit` (required), `--user` (`default`), `--comments` (`default`), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`savepoint delete`** Delete the savepoint.
+ <br />Options: `--commit` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`savepoint rollback`** Savepoint a commit.
+ <br />Options: `--savepoint` (required), `--sparkProperties`, `--sparkMaster`, `--lazyFailedWritesCleanPolicy` (`false`), `--sparkMemory` (`4G`)
+- **`savepoints show`** Show the savepoints.
+- **`show restore`** Show details of a restore instant.
+ <br />Options: `--instant` (required), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show restores`** List all restore instants.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeInflights` (`false`)
+- **`show rollback`** Show details of a rollback instant.
+ <br />Options: `--instant` (required), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show rollbacks`** List all rollback instants.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`upgrade table`** Upgrades a table.
+ <br />Options: `--toVersion`, `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+
+### Metadata table
+
+- **`metadata create`** Create the Metadata Table if it does not exist.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`)
+- **`metadata delete`** Remove the Metadata Table.
+ <br />Options: `--backup` (`true`)
+- **`metadata delete-record-index`** Delete the record index from Metadata Table.
+ <br />Options: `--backup` (`true`)
+- **`metadata init`** Update the metadata table from commits since the creation.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--readonly` (`false`)
+- **`metadata list-files`** Print a list of all files in a partition from the metadata.
+ <br />Options: `--partition`
+- **`metadata list-partitions`** List all partitions from metadata.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`)
+- **`metadata lookup-record-index`** Print Record index information for a record_key.
+ <br />Options: `--record_key` (required), `--partition_path` (required)
Review Comment:
**major:** `--partition_path` is optional in source (`MetadataCommand.java:398-399`, `defaultValue = ""`), and line 770 of this page already says so for a global RLI; `(required)` here contradicts both. The description is also cut at the first string literal, dropping the global-vs-partitioned rule. Could we take this?
```suggestion
- **`metadata lookup-record-index`** Print Record index information for a record_key. For global RLI, only record key is required. For partitioned RLI, both record key and partition path are required.
<br />Options: `--record_key` (required), `--partition_path`
```
##########
website/docs/cli.md:
##########
@@ -614,6 +671,20 @@ savepoints show
savepoint rollback --savepoint 20220128160245447 --sparkMaster local[2]
```
+A rollback to a savepoint writes a `restore` instant on the timeline. `show restores` lists them, and `show restore`
Review Comment:
**minor:** Not blocking, but the lists-versus-expands contrast is not real: `show restores` calls the same `populateOutputFromRestoreInstant` as `show restore` (`RestoresCommand.java:64-66` vs `:92`), emitting one row per reverted commit, and `--limit` caps rows, so one restore that reverted 12 commits fills the default page. Could we say both expand, with `show restore` scoped to one instant?
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
Review Comment:
**minor:** Not blocking, but `rename partition` only works when the table has a single partition field: `SparkMain#rewriteRecordsToNewPartition` calls `fieldIndex(getPartitionFieldProp())`, and that property is the comma-joined field list (`HoodieTableConfig.java:760-765`), so a year/month/day table throws before writing. `2026/08/26` reads as multi-level here; the only test uses one field holding `2016/03/15`. Could we add a sentence stating the constraint?
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
Review Comment:
**minor:** Not blocking. The write path has no widening predicate: `BaseHoodieWriteClient.java:1578` rejects any inequality and uses `isWiderThan` only to pick the error wording (`:1584`), and the CLI never calls `validateAgainstTableProperties`. The shared piece is `MetaFieldsMode#isWiderThan`. The source comment at `TableCommand.java:311-312` makes the same claim, so this was inherited. Could we cite the helper instead of the method?
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
Review Comment:
**minor:** Not blocking. `--show-rollback-info` also annotates non-rollback instants: `TimelineCommand.java:352-364` appends `Rolls back ...` for rollback instants and `Rolled back by ...` for any instant a rollback targeted, both into the Action column rather than a new one. Could we describe it as annotating both directions?
##########
website/docs/cli.md:
##########
@@ -253,29 +269,43 @@ hudi:trips->help
* create - Create a hoodie table if not present
* date - Displays the local date and time
Review Comment:
**minor:** Not blocking, but this block is Spring Shell 1.x output. hudi-cli has pinned 2.1.1 since #6489 (`pom.xml:261`), whose help template prints `AVAILABLE COMMANDS` with `name: description` per group and has no `!`, `//`, `;`, `clear`, `cls`, `date`, `script` or `system properties` built-ins. Extending it by 20 lines keeps a transcript no user can reproduce. Could we regenerate it from a 2.1.1 session, or drop it now that the Command reference supersedes it?
##########
website/docs/cli.md:
##########
@@ -160,6 +160,22 @@ Kerberos authentication success
If you see "Kerberos authentication success" in the command output, it means Kerberos authentication has been successful.
+**kerberos kdestroy** is the counterpart: it logs the current user out of the keytab and resets the cached
Review Comment:
**nit:** Feel free to ignore. The code logs out the login user, not the current user, and only when it holds Kerberos credentials (`KerberosAuthenticationCommand.java:70-76`). Could we say "logs the login user out of the keytab"?
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
Review Comment:
**nit:** Feel free to ignore. Rollbacks never appear in `commits show` regardless of state, since it filters to `getCommitsTimeline()` (`CommitsCommand.java:73`; commit, deltacommit, replacecommit and clustering only). Could we drop "that never completed" so the reason reads as action rather than state?
##########
website/docs/cli.md:
##########
@@ -698,6 +769,23 @@ hudi:trips->metadata lookup-record-index --record_key <key> --partition_path <pa
The `--partition_path` argument is optional for a global RLI (where record keys are unique across all partitions) and required for a partitioned RLI. If `--partition_path` is omitted for a partitioned RLI, the command will return an error. The output columns are `Record key`, `Partition path`, `File Id`, and `Instant time`.
+To drop the record index partition from the Metadata Table entirely, for example before rebuilding it:
+
+```shell
+hudi:trips->metadata delete-record-index --backup true
+Record Index has been deleted from the Metadata Table and backed up to /user/hive/warehouse/table1/.hoodie/.metadata_record_index_20260831090412345
+```
+
+`--backup` defaults to `true`. The backup is a rename rather than a copy, so it is cheap: the
+`.hoodie/metadata/record_index` partition is moved to `.hoodie/.metadata_record_index_<instantTime>`, where
+`<instantTime>` is the current instant in `yyyyMMddHHmmssSSS` form. Pass `--backup false` to delete the partition
+outright, in which case the output is just `Record Index has been deleted from the Metadata Table` and the only way
+back is to rebuild the index.
+
+Either way the command first flips the partition off in the table config, so readers stop consulting the index before
+the files go away. If the record index partition does not exist, nothing is deleted and the message still prints, with
Review Comment:
**nit:** Feel free to ignore. `null` is also printed when the backup rename fails or returns false (`HoodieTableMetadataUtil.java:1774-1781`); in that case the partition is neither backed up nor deleted, yet the success line still prints. Could we widen this sentence to "or if the rename fails"?
##########
website/docs/cli.md:
##########
@@ -160,6 +160,22 @@ Kerberos authentication success
If you see "Kerberos authentication success" in the command output, it means Kerberos authentication has been successful.
+**kerberos kdestroy** is the counterpart: it logs the current user out of the keytab and resets the cached
+`UserGroupInformation`, which is what you want before authenticating as a different principal in the same session.
+
+```shell
+hudi->kerberos kdestroy --krb5conf /etc/krb5.conf
Review Comment:
**nit:** Feel free to ignore. This block and the `metadata delete-record-index` one at line 774 use `shell`, while the other fourteen new transcript blocks use `java`. Could we pick one for the new blocks?
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (empty by default) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
+
+Every command `hudi-cli` exposes, grouped by area, with its options and their defaults. An option marked
+`(required)` has no default and must be supplied; a value in backticks after an option is its default.
+The sections above cover the commonly used ones in more depth.
+
+Some entries are aliases of the same command rather than distinct ones. `refresh`, `metadata refresh`,
+`commits refresh`, `cleans refresh` and `savepoints refresh` are five names for one method that reloads the table
+metadata, and `temp query` / `temp_query`, `temp delete` / `temp_delete` and `temps show` / `temps_show` are
+underscore and space spellings of the same three commands.
+
+### Table and session
+
+- **`cleans refresh`** Refresh table metadata.
+- **`commits refresh`** Refresh table metadata.
+- **`connect`** Connect to a hoodie table.
+ <br />Options: `--path` (required), `--eventuallyConsistent` (`false`), `--initialCheckIntervalMs` (`2000`), `--maxWaitIntervalMs` (`300000`), `--maxCheckIntervalMs` (`7`), `--timeGeneratorType` (`WAIT_TO_ADJUST_SKEW`), `--maxExpectedClockSkewMs` (`200`), `--useDefaultLockProvider` (`false`)
+- **`create`** Create a hoodie table if not present.
+ <br />Options: `--path` (required), `--tableName` (required), `--tableType` (`COPY_ON_WRITE`), `--archiveLogFolder`, `--tableVersion`, `--payloadClass` (`org.apache.hudi.common.model.HoodieAvroPayload`)
+- **`desc`** Describe Hoodie Table properties.
+- **`fetch table schema`** Fetches latest table schema.
+ <br />Options: `--outputFilePath`
+- **`kerberos kdestroy`** Destroy Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`)
+- **`kerberos kinit`** Perform Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`), `--principal` (required), `--keytab` (required)
+- **`metadata refresh`** Refresh table metadata.
+- **`refresh`** Refresh table metadata.
+- **`savepoints refresh`** Refresh table metadata.
+- **`set`** Set spark launcher env to cli.
+ <br />Options: `--conf` (required)
+- **`show env`** Show spark launcher env by key.
+ <br />Options: `--key` (required)
+- **`show envs all`** Show spark launcher envs.
+- **`table change-table-type`** Change hudi table type to target type: COW or MOR.
+ <br />Options: `--target-type` (required), `--enable-compaction` (`true`), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`table delete-configs`** Delete the supplied table configs from the table.
+ <br />Options: `--comma-separated-configs` (required)
+- **`table recover-configs`** Recover table configs, from update/delete that failed midway.
+- **`table set-meta-fields-mode`** Set hoodie.meta.fields.mode on an existing table. This is the sanctioned way to change.
+ <br />Options: `--target-mode` (required), `--force` (`false`)
+- **`table update-configs`** Update the table configs with configs with provided file.
+ <br />Options: `--props-file` (required)
+- **`utils loadClass`** Load a class.
+ <br />Options: `--class` (required)
+
+### Commits and the timeline
+
+- **`commit show_write_stats`** Show write stats of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showfiles`** Show file level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showpartitions`** Show partition level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commits compare`** Compare commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`commits show`** Show the commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`, `--includeArchivedTimeline` (`false`)
+- **`commits show_inflights`** Show inflight instants that are left longer than a certain duration.
+ <br />Options: `--lookbackInMins` (`0`)
+- **`commits showarchived`** Show the archived commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`commits sync`** Sync commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`diff file`** Check how file differs across range of commits.
+ <br />Options: `--fileId` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`diff partition`** Check how file differs across range of commits. It is meant to be used only for partitioned tables.
+ <br />Options: `--partitionPath` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`metadata timeline show active`** List all instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`metadata timeline show incomplete`** List all incomplete instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`show archived commit stats`** Read commits from archived files and show file group details.
+ <br />Options: `--archiveFolderPattern`, `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show archived commits`** Read commits from archived files and show details.
+ <br />Options: `--skipMetadata` (`true`), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`timeline show active`** List all instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--with-metadata-table` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`timeline show incomplete`** List all incomplete instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`trigger archival`** Trigger archival.
+ <br />Options: `--minCommits` (`20`), `--maxCommits` (`30`), `--commitsRetainedByCleaner` (`10`), `--enableMetadata` (`true`), `--sparkMemory` (`1G`), `--sparkMaster` (`local`)
+
+### Files, stats and log files
+
+- **`show fsview all`** Show entire file-system view.
+ <br />Options: `--pathRegex` (`*`), `--baseFileOnly` (`false`), `--maxInstant`, `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show fsview latest`** Show latest file-system view.
+ <br />Options: `--partitionPath`, `--baseFileOnly` (`false`), `--maxInstant`, `--merge` (`true`), `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile metadata`** Read commit metadata from log files.
+ <br />Options: `--logFilePathPattern` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile records`** Read records from log files.
+ <br />Options: `--limit` (`10`), `--logFilePathPattern` (required), `--mergeRecords` (`false`)
+- **`stats filesizes`** File Sizes. Display summary stats on sizes of files.
+ <br />Options: `--partitionPath` (`*/*/*`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`stats wa`** Write Amplification. Ratio of how many records were upserted to how many.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+
+### Table services
+
+- **`clean showpartitions`** Show partition level details of a clean.
+ <br />Options: `--clean` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`cleans run`** Run clean.
+ <br />Options: `--sparkMemory` (`4G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster`
+- **`cleans show`** Show the cleans.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--startTs`, `--endTs`, `--includeArchivedTimeline` (`false`), `--desc` (`false`), `--headeronly` (`false`)
+- **`clustering run`** Run Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--clusteringInstant`, `--propsFilePath`, `--hoodieConfigs`
+- **`clustering schedule`** Schedule Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`1g`), `--propsFilePath`, `--hoodieConfigs`
+- **`clustering scheduleAndExecute`** Run Clustering. Make a cluster plan first and execute that plan immediately.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`compaction repair`** Renames the files to make them consistent with the timeline as.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction run`** Run Compaction for given instant time.
+ <br />Options: `--parallelism` (`3`), `--schemaFilePath`, `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--compactionInstant`, `--propsFilePath`, `--hoodieConfigs`
+- **`compaction schedule`** Schedule Compaction.
+ <br />Options: `--sparkMemory` (`1G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster` (`local`)
+- **`compaction scheduleAndExecute`** Schedule compaction plan and execute this plan.
+ <br />Options: `--parallelism` (`3`), `--schemaFilePath`, `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`compaction show`** Shows compaction details for a specific compaction instant.
+ <br />Options: `--instant` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`compaction showarchived`** Shows compaction details for a specific compaction instant.
+ <br />Options: `--instant` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`compaction unschedule`** Unschedule Compaction.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--skipValidation` (`false`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction unscheduleFileId`** UnSchedule Compaction for a fileId.
+ <br />Options: `--fileId` (required), `--partitionPath`, `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--skipValidation` (`false`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction validate`** Validate Compaction.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compactions show all`** Shows all compactions that are in active timeline.
+ <br />Options: `--includeExtraMetadata` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compactions showarchived`** Shows compaction details for specified time window.
+ <br />Options: `--includeExtraMetadata` (`false`), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`marker delete`** Delete the marker.
+ <br />Options: `--commit` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`1G`)
+
+### Rollback, savepoint, restore and repair
+
+- **`commit rollback`** Rollback a commit.
+ <br />Options: `--commit` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`), `--rollbackUsingMarkers` (`false`)
+- **`downgrade table`** Downgrades a table.
+ <br />Options: `--toVersion`, `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`rename partition`** Rename partition. Usage: rename partition --oldPartition <oldPartition> --newPartition <newPartition>.
+ <br />Options: `--oldPartition` (required), `--newPartition` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`repair addpartitionmeta`** Add partition metadata to a table, if not present.
+ <br />Options: `--dryrun` (`true`)
+- **`repair corrupted clean files`** Repair corrupted clean files.
+- **`repair deduplicate`** De-duplicate a partition path contains duplicates & produce repaired files to replace with.
+ <br />Options: `--duplicatedPartitionPath`, `--repairedOutputPath` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`), `--dryrun` (`true`), `--dedupeType` (`insert_type`)
+- **`repair deprecated partition`** Repair deprecated partition ("default"). Re-writes data from the deprecated partition into.
Review Comment:
**minor:** Not blocking. Seven reference descriptions end where the first string literal of a `+`-concatenated `value=` ends (lines 1102, 1107, 1163, 1180, 1218, 1220, 1257). This one loses the actual target, which the help line at 305 renders in full. Could we complete them, e.g.:
```suggestion
- **`repair deprecated partition`** Repair deprecated partition ("default"). Re-writes data from the deprecated partition into `__HIVE_DEFAULT_PARTITION__`.
```
##########
website/docs/cli.md:
##########
@@ -291,6 +321,8 @@ hudi:trips->help
* show fsview latest - Show latest file-system view
Review Comment:
**nit:** Feel free to ignore, but line 317 just above is a pre-existing copy of the `show archived commits` line: the source string is "Read commits from archived files and show file group details" (`ArchivedCommitsCommand.java:99`). Could we fix it while the block is open?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(cli): document all 106 CLI commands [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on code in PR #19799:
URL: https://github.com/apache/hudi/pull/19799#discussion_r3946470246
##########
website/docs/cli.md:
##########
@@ -253,29 +269,43 @@ hudi:trips->help
* create - Create a hoodie table if not present
* date - Displays the local date and time
Review Comment:
Went with dropping it. `spring.shell.version` is 2.1.1 at `pom.xml:261`, so the block was unreproducible, and the Command reference this PR adds covers the same ground. Replaced with three lines pointing at it.
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
Review Comment:
Added the constraint. Confirmed `rewriteRecordsToNewPartition` calls `structType.fieldIndex(partitionFieldProp)` and that `getPartitionFieldProp` joins the field list, so a multi-field table throws before writing.
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
Review Comment:
Fixed, it now cites `MetaFieldsMode#isWiderThan`. Also noted the write path is stricter rather than equivalent, since `BaseHoodieWriteClient:1578` rejects any inequality and uses `isWiderThan` only for the error wording.
##########
website/docs/cli.md:
##########
@@ -614,6 +671,20 @@ savepoints show
savepoint rollback --savepoint 20220128160245447 --sparkMaster local[2]
```
+A rollback to a savepoint writes a `restore` instant on the timeline. `show restores` lists them, and `show restore`
Review Comment:
Corrected. Both call `populateOutputFromRestoreInstant`, so both expand; the text now says `show restore` is the same view scoped to one instant, and notes `--limit` caps rows rather than restores.
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
Review Comment:
Reworded to describe both directions, matching `getRollbackInfoString`.
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (empty by default) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
+
+Every command `hudi-cli` exposes, grouped by area, with its options and their defaults. An option marked
+`(required)` has no default and must be supplied; a value in backticks after an option is its default.
+The sections above cover the commonly used ones in more depth.
+
+Some entries are aliases of the same command rather than distinct ones. `refresh`, `metadata refresh`,
+`commits refresh`, `cleans refresh` and `savepoints refresh` are five names for one method that reloads the table
+metadata, and `temp query` / `temp_query`, `temp delete` / `temp_delete` and `temps show` / `temps_show` are
+underscore and space spellings of the same three commands.
+
+### Table and session
+
+- **`cleans refresh`** Refresh table metadata.
+- **`commits refresh`** Refresh table metadata.
+- **`connect`** Connect to a hoodie table.
+ <br />Options: `--path` (required), `--eventuallyConsistent` (`false`), `--initialCheckIntervalMs` (`2000`), `--maxWaitIntervalMs` (`300000`), `--maxCheckIntervalMs` (`7`), `--timeGeneratorType` (`WAIT_TO_ADJUST_SKEW`), `--maxExpectedClockSkewMs` (`200`), `--useDefaultLockProvider` (`false`)
+- **`create`** Create a hoodie table if not present.
+ <br />Options: `--path` (required), `--tableName` (required), `--tableType` (`COPY_ON_WRITE`), `--archiveLogFolder`, `--tableVersion`, `--payloadClass` (`org.apache.hudi.common.model.HoodieAvroPayload`)
+- **`desc`** Describe Hoodie Table properties.
+- **`fetch table schema`** Fetches latest table schema.
+ <br />Options: `--outputFilePath`
+- **`kerberos kdestroy`** Destroy Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`)
+- **`kerberos kinit`** Perform Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`), `--principal` (required), `--keytab` (required)
+- **`metadata refresh`** Refresh table metadata.
+- **`refresh`** Refresh table metadata.
+- **`savepoints refresh`** Refresh table metadata.
+- **`set`** Set spark launcher env to cli.
+ <br />Options: `--conf` (required)
+- **`show env`** Show spark launcher env by key.
+ <br />Options: `--key` (required)
+- **`show envs all`** Show spark launcher envs.
+- **`table change-table-type`** Change hudi table type to target type: COW or MOR.
+ <br />Options: `--target-type` (required), `--enable-compaction` (`true`), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`table delete-configs`** Delete the supplied table configs from the table.
+ <br />Options: `--comma-separated-configs` (required)
+- **`table recover-configs`** Recover table configs, from update/delete that failed midway.
+- **`table set-meta-fields-mode`** Set hoodie.meta.fields.mode on an existing table. This is the sanctioned way to change.
+ <br />Options: `--target-mode` (required), `--force` (`false`)
+- **`table update-configs`** Update the table configs with configs with provided file.
+ <br />Options: `--props-file` (required)
+- **`utils loadClass`** Load a class.
+ <br />Options: `--class` (required)
+
+### Commits and the timeline
+
+- **`commit show_write_stats`** Show write stats of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showfiles`** Show file level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showpartitions`** Show partition level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commits compare`** Compare commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`commits show`** Show the commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`, `--includeArchivedTimeline` (`false`)
+- **`commits show_inflights`** Show inflight instants that are left longer than a certain duration.
+ <br />Options: `--lookbackInMins` (`0`)
+- **`commits showarchived`** Show the archived commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`commits sync`** Sync commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`diff file`** Check how file differs across range of commits.
+ <br />Options: `--fileId` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`diff partition`** Check how file differs across range of commits. It is meant to be used only for partitioned tables.
+ <br />Options: `--partitionPath` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`metadata timeline show active`** List all instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`metadata timeline show incomplete`** List all incomplete instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`show archived commit stats`** Read commits from archived files and show file group details.
+ <br />Options: `--archiveFolderPattern`, `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show archived commits`** Read commits from archived files and show details.
+ <br />Options: `--skipMetadata` (`true`), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`timeline show active`** List all instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--with-metadata-table` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`timeline show incomplete`** List all incomplete instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`trigger archival`** Trigger archival.
+ <br />Options: `--minCommits` (`20`), `--maxCommits` (`30`), `--commitsRetainedByCleaner` (`10`), `--enableMetadata` (`true`), `--sparkMemory` (`1G`), `--sparkMaster` (`local`)
+
+### Files, stats and log files
+
+- **`show fsview all`** Show entire file-system view.
+ <br />Options: `--pathRegex` (`*`), `--baseFileOnly` (`false`), `--maxInstant`, `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show fsview latest`** Show latest file-system view.
+ <br />Options: `--partitionPath`, `--baseFileOnly` (`false`), `--maxInstant`, `--merge` (`true`), `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile metadata`** Read commit metadata from log files.
+ <br />Options: `--logFilePathPattern` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile records`** Read records from log files.
+ <br />Options: `--limit` (`10`), `--logFilePathPattern` (required), `--mergeRecords` (`false`)
+- **`stats filesizes`** File Sizes. Display summary stats on sizes of files.
+ <br />Options: `--partitionPath` (`*/*/*`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`stats wa`** Write Amplification. Ratio of how many records were upserted to how many.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+
+### Table services
+
+- **`clean showpartitions`** Show partition level details of a clean.
+ <br />Options: `--clean` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`cleans run`** Run clean.
+ <br />Options: `--sparkMemory` (`4G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster`
+- **`cleans show`** Show the cleans.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--startTs`, `--endTs`, `--includeArchivedTimeline` (`false`), `--desc` (`false`), `--headeronly` (`false`)
+- **`clustering run`** Run Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--clusteringInstant`, `--propsFilePath`, `--hoodieConfigs`
+- **`clustering schedule`** Schedule Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`1g`), `--propsFilePath`, `--hoodieConfigs`
+- **`clustering scheduleAndExecute`** Run Clustering. Make a cluster plan first and execute that plan immediately.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`compaction repair`** Renames the files to make them consistent with the timeline as.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction run`** Run Compaction for given instant time.
+ <br />Options: `--parallelism` (`3`), `--schemaFilePath`, `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--compactionInstant`, `--propsFilePath`, `--hoodieConfigs`
+- **`compaction schedule`** Schedule Compaction.
+ <br />Options: `--sparkMemory` (`1G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster` (`local`)
+- **`compaction scheduleAndExecute`** Schedule compaction plan and execute this plan.
+ <br />Options: `--parallelism` (`3`), `--schemaFilePath`, `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`compaction show`** Shows compaction details for a specific compaction instant.
+ <br />Options: `--instant` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`compaction showarchived`** Shows compaction details for a specific compaction instant.
+ <br />Options: `--instant` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`compaction unschedule`** Unschedule Compaction.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--skipValidation` (`false`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction unscheduleFileId`** UnSchedule Compaction for a fileId.
+ <br />Options: `--fileId` (required), `--partitionPath`, `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--skipValidation` (`false`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction validate`** Validate Compaction.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compactions show all`** Shows all compactions that are in active timeline.
+ <br />Options: `--includeExtraMetadata` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compactions showarchived`** Shows compaction details for specified time window.
+ <br />Options: `--includeExtraMetadata` (`false`), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`marker delete`** Delete the marker.
+ <br />Options: `--commit` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`1G`)
+
+### Rollback, savepoint, restore and repair
+
+- **`commit rollback`** Rollback a commit.
+ <br />Options: `--commit` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`), `--rollbackUsingMarkers` (`false`)
+- **`downgrade table`** Downgrades a table.
+ <br />Options: `--toVersion`, `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`rename partition`** Rename partition. Usage: rename partition --oldPartition <oldPartition> --newPartition <newPartition>.
+ <br />Options: `--oldPartition` (required), `--newPartition` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`repair addpartitionmeta`** Add partition metadata to a table, if not present.
+ <br />Options: `--dryrun` (`true`)
+- **`repair corrupted clean files`** Repair corrupted clean files.
+- **`repair deduplicate`** De-duplicate a partition path contains duplicates & produce repaired files to replace with.
+ <br />Options: `--duplicatedPartitionPath`, `--repairedOutputPath` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`), `--dryrun` (`true`), `--dedupeType` (`insert_type`)
+- **`repair deprecated partition`** Repair deprecated partition ("default"). Re-writes data from the deprecated partition into.
Review Comment:
Completed five of them: `repair deprecated partition` plus `compaction repair`, `repair migrate-partition-meta`, `stats wa` and `table change-table-type`, each against the concatenated source string.
##########
website/docs/cli.md:
##########
@@ -291,6 +321,8 @@ hudi:trips->help
* show fsview latest - Show latest file-system view
Review Comment:
Moot now, that line went with the `help` transcript.
##########
website/docs/cli.md:
##########
@@ -160,6 +160,22 @@ Kerberos authentication success
If you see "Kerberos authentication success" in the command output, it means Kerberos authentication has been successful.
+**kerberos kdestroy** is the counterpart: it logs the current user out of the keytab and resets the cached
Review Comment:
Reworded to the login user, and noted it only acts when that user holds Kerberos credentials.
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
Review Comment:
Dropped the phrase, and said instead that `commits show` only covers commit, deltacommit, replacecommit and clustering, so a rollback never appears in any state.
##########
website/docs/cli.md:
##########
@@ -698,6 +769,23 @@ hudi:trips->metadata lookup-record-index --record_key <key> --partition_path <pa
The `--partition_path` argument is optional for a global RLI (where record keys are unique across all partitions) and required for a partitioned RLI. If `--partition_path` is omitted for a partitioned RLI, the command will return an error. The output columns are `Record key`, `Partition path`, `File Id`, and `Instant time`.
+To drop the record index partition from the Metadata Table entirely, for example before rebuilding it:
+
+```shell
+hudi:trips->metadata delete-record-index --backup true
+Record Index has been deleted from the Metadata Table and backed up to /user/hive/warehouse/table1/.hoodie/.metadata_record_index_20260831090412345
+```
+
+`--backup` defaults to `true`. The backup is a rename rather than a copy, so it is cheap: the
+`.hoodie/metadata/record_index` partition is moved to `.hoodie/.metadata_record_index_<instantTime>`, where
+`<instantTime>` is the current instant in `yyyyMMddHHmmssSSS` form. Pass `--backup false` to delete the partition
+outright, in which case the output is just `Record Index has been deleted from the Metadata Table` and the only way
+back is to rebuild the index.
+
+Either way the command first flips the partition off in the table config, so readers stop consulting the index before
+the files go away. If the record index partition does not exist, nothing is deleted and the message still prints, with
Review Comment:
Widened, and added that a `null` there is not by itself confirmation the partition was absent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(cli): document all 106 CLI commands [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #19799:
URL: https://github.com/apache/hudi/pull/19799#discussion_r3916375290
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (empty by default) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
+
+Every command `hudi-cli` exposes, grouped by area, with its options and their defaults. An option marked
+`(required)` has no default and must be supplied; a value in backticks after an option is its default.
+The sections above cover the commonly used ones in more depth.
+
+Some entries are aliases of the same command rather than distinct ones. `refresh`, `metadata refresh`,
+`commits refresh`, `cleans refresh` and `savepoints refresh` are five names for one method that reloads the table
+metadata, and `temp query` / `temp_query`, `temp delete` / `temp_delete` and `temps show` / `temps_show` are
+underscore and space spellings of the same three commands.
+
+### Table and session
+
+- **`cleans refresh`** Refresh table metadata.
+- **`commits refresh`** Refresh table metadata.
+- **`connect`** Connect to a hoodie table.
+ <br />Options: `--path` (required), `--eventuallyConsistent` (`false`), `--initialCheckIntervalMs` (`2000`), `--maxWaitIntervalMs` (`300000`), `--maxCheckIntervalMs` (`7`), `--timeGeneratorType` (`WAIT_TO_ADJUST_SKEW`), `--maxExpectedClockSkewMs` (`200`), `--useDefaultLockProvider` (`false`)
+- **`create`** Create a hoodie table if not present.
+ <br />Options: `--path` (required), `--tableName` (required), `--tableType` (`COPY_ON_WRITE`), `--archiveLogFolder`, `--tableVersion`, `--payloadClass` (`org.apache.hudi.common.model.HoodieAvroPayload`)
+- **`desc`** Describe Hoodie Table properties.
+- **`fetch table schema`** Fetches latest table schema.
+ <br />Options: `--outputFilePath`
+- **`kerberos kdestroy`** Destroy Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`)
+- **`kerberos kinit`** Perform Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`), `--principal` (required), `--keytab` (required)
+- **`metadata refresh`** Refresh table metadata.
+- **`refresh`** Refresh table metadata.
+- **`savepoints refresh`** Refresh table metadata.
+- **`set`** Set spark launcher env to cli.
+ <br />Options: `--conf` (required)
+- **`show env`** Show spark launcher env by key.
+ <br />Options: `--key` (required)
+- **`show envs all`** Show spark launcher envs.
+- **`table change-table-type`** Change hudi table type to target type: COW or MOR.
+ <br />Options: `--target-type` (required), `--enable-compaction` (`true`), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`table delete-configs`** Delete the supplied table configs from the table.
+ <br />Options: `--comma-separated-configs` (required)
+- **`table recover-configs`** Recover table configs, from update/delete that failed midway.
+- **`table set-meta-fields-mode`** Set hoodie.meta.fields.mode on an existing table. This is the sanctioned way to change.
+ <br />Options: `--target-mode` (required), `--force` (`false`)
+- **`table update-configs`** Update the table configs with configs with provided file.
+ <br />Options: `--props-file` (required)
+- **`utils loadClass`** Load a class.
+ <br />Options: `--class` (required)
+
+### Commits and the timeline
+
+- **`commit show_write_stats`** Show write stats of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showfiles`** Show file level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showpartitions`** Show partition level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commits compare`** Compare commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`commits show`** Show the commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`, `--includeArchivedTimeline` (`false`)
+- **`commits show_inflights`** Show inflight instants that are left longer than a certain duration.
+ <br />Options: `--lookbackInMins` (`0`)
+- **`commits showarchived`** Show the archived commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`commits sync`** Sync commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`diff file`** Check how file differs across range of commits.
+ <br />Options: `--fileId` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`diff partition`** Check how file differs across range of commits. It is meant to be used only for partitioned tables.
+ <br />Options: `--partitionPath` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`metadata timeline show active`** List all instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`metadata timeline show incomplete`** List all incomplete instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`show archived commit stats`** Read commits from archived files and show file group details.
+ <br />Options: `--archiveFolderPattern`, `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show archived commits`** Read commits from archived files and show details.
+ <br />Options: `--skipMetadata` (`true`), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`timeline show active`** List all instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--with-metadata-table` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`timeline show incomplete`** List all incomplete instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`trigger archival`** Trigger archival.
+ <br />Options: `--minCommits` (`20`), `--maxCommits` (`30`), `--commitsRetainedByCleaner` (`10`), `--enableMetadata` (`true`), `--sparkMemory` (`1G`), `--sparkMaster` (`local`)
+
+### Files, stats and log files
+
+- **`show fsview all`** Show entire file-system view.
+ <br />Options: `--pathRegex` (`*`), `--baseFileOnly` (`false`), `--maxInstant`, `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show fsview latest`** Show latest file-system view.
+ <br />Options: `--partitionPath`, `--baseFileOnly` (`false`), `--maxInstant`, `--merge` (`true`), `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile metadata`** Read commit metadata from log files.
+ <br />Options: `--logFilePathPattern` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile records`** Read records from log files.
+ <br />Options: `--limit` (`10`), `--logFilePathPattern` (required), `--mergeRecords` (`false`)
+- **`stats filesizes`** File Sizes. Display summary stats on sizes of files.
+ <br />Options: `--partitionPath` (`*/*/*`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`stats wa`** Write Amplification. Ratio of how many records were upserted to how many.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+
+### Table services
+
+- **`clean showpartitions`** Show partition level details of a clean.
+ <br />Options: `--clean` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`cleans run`** Run clean.
+ <br />Options: `--sparkMemory` (`4G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster`
+- **`cleans show`** Show the cleans.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--startTs`, `--endTs`, `--includeArchivedTimeline` (`false`), `--desc` (`false`), `--headeronly` (`false`)
+- **`clustering run`** Run Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--clusteringInstant`, `--propsFilePath`, `--hoodieConfigs`
Review Comment:
**minor:** Not blocking, but the same trap is wider than these six entries. `SparkUtil.getDefaultConf` (`SparkUtil.java:89-95`) treats a blank master as absent, so the twelve `--sparkMaster` options rendered bare in this reference (`defaultValue = ""`) and the "empty by default" wording at line 998 also resolve to `yarn`. The legend at lines 1070-1071 has no state for the 119 options rendered without a value. Could we add a legend clause for unset defaults and one sentence that an unset `--sparkMaster` means `yarn`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(cli): document the undocumented CLI commands and how to set Hudi properties [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #19799:
URL: https://github.com/apache/hudi/pull/19799#issuecomment-5473778124
Is this a 2 part PR or? The description says part of. If this is a 2 part PR, please make it clear in the title this is Part 1 of N, where N should be clearly defined.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(cli): document all 106 CLI commands [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on code in PR #19799:
URL: https://github.com/apache/hudi/pull/19799#discussion_r3946469667
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (empty by default) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
+
+Every command `hudi-cli` exposes, grouped by area, with its options and their defaults. An option marked
+`(required)` has no default and must be supplied; a value in backticks after an option is its default.
+The sections above cover the commonly used ones in more depth.
+
+Some entries are aliases of the same command rather than distinct ones. `refresh`, `metadata refresh`,
+`commits refresh`, `cleans refresh` and `savepoints refresh` are five names for one method that reloads the table
+metadata, and `temp query` / `temp_query`, `temp delete` / `temp_delete` and `temps show` / `temps_show` are
+underscore and space spellings of the same three commands.
+
+### Table and session
+
+- **`cleans refresh`** Refresh table metadata.
+- **`commits refresh`** Refresh table metadata.
+- **`connect`** Connect to a hoodie table.
+ <br />Options: `--path` (required), `--eventuallyConsistent` (`false`), `--initialCheckIntervalMs` (`2000`), `--maxWaitIntervalMs` (`300000`), `--maxCheckIntervalMs` (`7`), `--timeGeneratorType` (`WAIT_TO_ADJUST_SKEW`), `--maxExpectedClockSkewMs` (`200`), `--useDefaultLockProvider` (`false`)
+- **`create`** Create a hoodie table if not present.
+ <br />Options: `--path` (required), `--tableName` (required), `--tableType` (`COPY_ON_WRITE`), `--archiveLogFolder`, `--tableVersion`, `--payloadClass` (`org.apache.hudi.common.model.HoodieAvroPayload`)
+- **`desc`** Describe Hoodie Table properties.
+- **`fetch table schema`** Fetches latest table schema.
+ <br />Options: `--outputFilePath`
+- **`kerberos kdestroy`** Destroy Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`)
+- **`kerberos kinit`** Perform Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`), `--principal` (required), `--keytab` (required)
+- **`metadata refresh`** Refresh table metadata.
+- **`refresh`** Refresh table metadata.
+- **`savepoints refresh`** Refresh table metadata.
+- **`set`** Set spark launcher env to cli.
+ <br />Options: `--conf` (required)
+- **`show env`** Show spark launcher env by key.
+ <br />Options: `--key` (required)
+- **`show envs all`** Show spark launcher envs.
+- **`table change-table-type`** Change hudi table type to target type: COW or MOR.
+ <br />Options: `--target-type` (required), `--enable-compaction` (`true`), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`table delete-configs`** Delete the supplied table configs from the table.
+ <br />Options: `--comma-separated-configs` (required)
+- **`table recover-configs`** Recover table configs, from update/delete that failed midway.
+- **`table set-meta-fields-mode`** Set hoodie.meta.fields.mode on an existing table. This is the sanctioned way to change.
+ <br />Options: `--target-mode` (required), `--force` (`false`)
+- **`table update-configs`** Update the table configs with configs with provided file.
+ <br />Options: `--props-file` (required)
+- **`utils loadClass`** Load a class.
+ <br />Options: `--class` (required)
+
+### Commits and the timeline
+
+- **`commit show_write_stats`** Show write stats of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showfiles`** Show file level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showpartitions`** Show partition level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commits compare`** Compare commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`commits show`** Show the commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`, `--includeArchivedTimeline` (`false`)
+- **`commits show_inflights`** Show inflight instants that are left longer than a certain duration.
+ <br />Options: `--lookbackInMins` (`0`)
+- **`commits showarchived`** Show the archived commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`commits sync`** Sync commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`diff file`** Check how file differs across range of commits.
+ <br />Options: `--fileId` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`diff partition`** Check how file differs across range of commits. It is meant to be used only for partitioned tables.
+ <br />Options: `--partitionPath` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`metadata timeline show active`** List all instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`metadata timeline show incomplete`** List all incomplete instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`show archived commit stats`** Read commits from archived files and show file group details.
+ <br />Options: `--archiveFolderPattern`, `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show archived commits`** Read commits from archived files and show details.
+ <br />Options: `--skipMetadata` (`true`), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`timeline show active`** List all instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--with-metadata-table` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`timeline show incomplete`** List all incomplete instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`trigger archival`** Trigger archival.
+ <br />Options: `--minCommits` (`20`), `--maxCommits` (`30`), `--commitsRetainedByCleaner` (`10`), `--enableMetadata` (`true`), `--sparkMemory` (`1G`), `--sparkMaster` (`local`)
+
+### Files, stats and log files
+
+- **`show fsview all`** Show entire file-system view.
+ <br />Options: `--pathRegex` (`*`), `--baseFileOnly` (`false`), `--maxInstant`, `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show fsview latest`** Show latest file-system view.
+ <br />Options: `--partitionPath`, `--baseFileOnly` (`false`), `--maxInstant`, `--merge` (`true`), `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile metadata`** Read commit metadata from log files.
+ <br />Options: `--logFilePathPattern` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile records`** Read records from log files.
+ <br />Options: `--limit` (`10`), `--logFilePathPattern` (required), `--mergeRecords` (`false`)
+- **`stats filesizes`** File Sizes. Display summary stats on sizes of files.
+ <br />Options: `--partitionPath` (`*/*/*`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`stats wa`** Write Amplification. Ratio of how many records were upserted to how many.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+
+### Table services
+
+- **`clean showpartitions`** Show partition level details of a clean.
+ <br />Options: `--clean` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`cleans run`** Run clean.
+ <br />Options: `--sparkMemory` (`4G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster`
+- **`cleans show`** Show the cleans.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--startTs`, `--endTs`, `--includeArchivedTimeline` (`false`), `--desc` (`false`), `--headeronly` (`false`)
+- **`clustering run`** Run Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--clusteringInstant`, `--propsFilePath`, `--hoodieConfigs`
Review Comment:
Fixed, all six now read `yarn` rather than the raw constant. Also added the wider note voonhous raised below, since an unset `--sparkMaster` resolves to `yarn` too.
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (empty by default) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
+
+Every command `hudi-cli` exposes, grouped by area, with its options and their defaults. An option marked
+`(required)` has no default and must be supplied; a value in backticks after an option is its default.
+The sections above cover the commonly used ones in more depth.
+
+Some entries are aliases of the same command rather than distinct ones. `refresh`, `metadata refresh`,
+`commits refresh`, `cleans refresh` and `savepoints refresh` are five names for one method that reloads the table
+metadata, and `temp query` / `temp_query`, `temp delete` / `temp_delete` and `temps show` / `temps_show` are
+underscore and space spellings of the same three commands.
+
+### Table and session
+
+- **`cleans refresh`** Refresh table metadata.
+- **`commits refresh`** Refresh table metadata.
+- **`connect`** Connect to a hoodie table.
+ <br />Options: `--path` (required), `--eventuallyConsistent` (`false`), `--initialCheckIntervalMs` (`2000`), `--maxWaitIntervalMs` (`300000`), `--maxCheckIntervalMs` (`7`), `--timeGeneratorType` (`WAIT_TO_ADJUST_SKEW`), `--maxExpectedClockSkewMs` (`200`), `--useDefaultLockProvider` (`false`)
+- **`create`** Create a hoodie table if not present.
+ <br />Options: `--path` (required), `--tableName` (required), `--tableType` (`COPY_ON_WRITE`), `--archiveLogFolder`, `--tableVersion`, `--payloadClass` (`org.apache.hudi.common.model.HoodieAvroPayload`)
+- **`desc`** Describe Hoodie Table properties.
+- **`fetch table schema`** Fetches latest table schema.
+ <br />Options: `--outputFilePath`
+- **`kerberos kdestroy`** Destroy Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`)
+- **`kerberos kinit`** Perform Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`), `--principal` (required), `--keytab` (required)
+- **`metadata refresh`** Refresh table metadata.
+- **`refresh`** Refresh table metadata.
+- **`savepoints refresh`** Refresh table metadata.
+- **`set`** Set spark launcher env to cli.
+ <br />Options: `--conf` (required)
+- **`show env`** Show spark launcher env by key.
+ <br />Options: `--key` (required)
+- **`show envs all`** Show spark launcher envs.
+- **`table change-table-type`** Change hudi table type to target type: COW or MOR.
+ <br />Options: `--target-type` (required), `--enable-compaction` (`true`), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`table delete-configs`** Delete the supplied table configs from the table.
+ <br />Options: `--comma-separated-configs` (required)
+- **`table recover-configs`** Recover table configs, from update/delete that failed midway.
+- **`table set-meta-fields-mode`** Set hoodie.meta.fields.mode on an existing table. This is the sanctioned way to change.
+ <br />Options: `--target-mode` (required), `--force` (`false`)
+- **`table update-configs`** Update the table configs with configs with provided file.
+ <br />Options: `--props-file` (required)
+- **`utils loadClass`** Load a class.
+ <br />Options: `--class` (required)
+
+### Commits and the timeline
+
+- **`commit show_write_stats`** Show write stats of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showfiles`** Show file level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showpartitions`** Show partition level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commits compare`** Compare commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`commits show`** Show the commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`, `--includeArchivedTimeline` (`false`)
+- **`commits show_inflights`** Show inflight instants that are left longer than a certain duration.
+ <br />Options: `--lookbackInMins` (`0`)
+- **`commits showarchived`** Show the archived commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`commits sync`** Sync commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`diff file`** Check how file differs across range of commits.
+ <br />Options: `--fileId` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`diff partition`** Check how file differs across range of commits. It is meant to be used only for partitioned tables.
+ <br />Options: `--partitionPath` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`metadata timeline show active`** List all instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`metadata timeline show incomplete`** List all incomplete instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`show archived commit stats`** Read commits from archived files and show file group details.
+ <br />Options: `--archiveFolderPattern`, `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show archived commits`** Read commits from archived files and show details.
+ <br />Options: `--skipMetadata` (`true`), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`timeline show active`** List all instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--with-metadata-table` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`timeline show incomplete`** List all incomplete instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`trigger archival`** Trigger archival.
+ <br />Options: `--minCommits` (`20`), `--maxCommits` (`30`), `--commitsRetainedByCleaner` (`10`), `--enableMetadata` (`true`), `--sparkMemory` (`1G`), `--sparkMaster` (`local`)
+
+### Files, stats and log files
+
+- **`show fsview all`** Show entire file-system view.
+ <br />Options: `--pathRegex` (`*`), `--baseFileOnly` (`false`), `--maxInstant`, `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show fsview latest`** Show latest file-system view.
+ <br />Options: `--partitionPath`, `--baseFileOnly` (`false`), `--maxInstant`, `--merge` (`true`), `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile metadata`** Read commit metadata from log files.
+ <br />Options: `--logFilePathPattern` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile records`** Read records from log files.
+ <br />Options: `--limit` (`10`), `--logFilePathPattern` (required), `--mergeRecords` (`false`)
+- **`stats filesizes`** File Sizes. Display summary stats on sizes of files.
+ <br />Options: `--partitionPath` (`*/*/*`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`stats wa`** Write Amplification. Ratio of how many records were upserted to how many.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+
+### Table services
+
+- **`clean showpartitions`** Show partition level details of a clean.
+ <br />Options: `--clean` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`cleans run`** Run clean.
+ <br />Options: `--sparkMemory` (`4G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster`
+- **`cleans show`** Show the cleans.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--startTs`, `--endTs`, `--includeArchivedTimeline` (`false`), `--desc` (`false`), `--headeronly` (`false`)
+- **`clustering run`** Run Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--clusteringInstant`, `--propsFilePath`, `--hoodieConfigs`
Review Comment:
Added both: the legend now has a clause for options rendered without a value, and a note that an unset or blank `--sparkMaster` resolves to `yarn`, since `getDefaultConf` treats a blank as absent. Confirmed against `SparkUtil.java:89-95`.
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (empty by default) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
Review Comment:
Mirrored into `versioned_docs/version-1.2.0/cli.md`. Confirmed the mapping you cite - `1.2.0` to path `""`, `current` to `next` - so none of this was reachable at `/docs/cli`. The 1.2.0 copy drops the Meta Fields Mode section and its reference entry, and uses `commits show_infights`; I verified both against `release-1.2.0`, and that every other command documented here exists there.
##########
website/docs/cli.md:
##########
@@ -698,6 +769,23 @@ hudi:trips->metadata lookup-record-index --record_key <key> --partition_path <pa
The `--partition_path` argument is optional for a global RLI (where record keys are unique across all partitions) and required for a partitioned RLI. If `--partition_path` is omitted for a partitioned RLI, the command will return an error. The output columns are `Record key`, `Partition path`, `File Id`, and `Instant time`.
+To drop the record index partition from the Metadata Table entirely, for example before rebuilding it:
+
+```shell
+hudi:trips->metadata delete-record-index --backup true
Review Comment:
Noted the prerequisite. Confirmed the mechanism: `initJavaSparkContext` is only called from `metadata create`, `init` and `list-partitions`, so `deleteRecordIndex` at `:156` has no `jsc` after a bare `connect`. The transcript now runs `metadata list-partitions` first. Happy to follow up with the `HoodieLocalEngineContext` fix on master separately, since `deleteMetadataTablePartition` never reads the context.
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (empty by default) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
+
+Every command `hudi-cli` exposes, grouped by area, with its options and their defaults. An option marked
+`(required)` has no default and must be supplied; a value in backticks after an option is its default.
+The sections above cover the commonly used ones in more depth.
+
+Some entries are aliases of the same command rather than distinct ones. `refresh`, `metadata refresh`,
+`commits refresh`, `cleans refresh` and `savepoints refresh` are five names for one method that reloads the table
+metadata, and `temp query` / `temp_query`, `temp delete` / `temp_delete` and `temps show` / `temps_show` are
+underscore and space spellings of the same three commands.
+
+### Table and session
+
+- **`cleans refresh`** Refresh table metadata.
+- **`commits refresh`** Refresh table metadata.
+- **`connect`** Connect to a hoodie table.
+ <br />Options: `--path` (required), `--eventuallyConsistent` (`false`), `--initialCheckIntervalMs` (`2000`), `--maxWaitIntervalMs` (`300000`), `--maxCheckIntervalMs` (`7`), `--timeGeneratorType` (`WAIT_TO_ADJUST_SKEW`), `--maxExpectedClockSkewMs` (`200`), `--useDefaultLockProvider` (`false`)
+- **`create`** Create a hoodie table if not present.
+ <br />Options: `--path` (required), `--tableName` (required), `--tableType` (`COPY_ON_WRITE`), `--archiveLogFolder`, `--tableVersion`, `--payloadClass` (`org.apache.hudi.common.model.HoodieAvroPayload`)
+- **`desc`** Describe Hoodie Table properties.
+- **`fetch table schema`** Fetches latest table schema.
+ <br />Options: `--outputFilePath`
+- **`kerberos kdestroy`** Destroy Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`)
+- **`kerberos kinit`** Perform Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`), `--principal` (required), `--keytab` (required)
+- **`metadata refresh`** Refresh table metadata.
+- **`refresh`** Refresh table metadata.
+- **`savepoints refresh`** Refresh table metadata.
+- **`set`** Set spark launcher env to cli.
+ <br />Options: `--conf` (required)
+- **`show env`** Show spark launcher env by key.
+ <br />Options: `--key` (required)
+- **`show envs all`** Show spark launcher envs.
+- **`table change-table-type`** Change hudi table type to target type: COW or MOR.
+ <br />Options: `--target-type` (required), `--enable-compaction` (`true`), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`table delete-configs`** Delete the supplied table configs from the table.
+ <br />Options: `--comma-separated-configs` (required)
+- **`table recover-configs`** Recover table configs, from update/delete that failed midway.
+- **`table set-meta-fields-mode`** Set hoodie.meta.fields.mode on an existing table. This is the sanctioned way to change.
+ <br />Options: `--target-mode` (required), `--force` (`false`)
+- **`table update-configs`** Update the table configs with configs with provided file.
+ <br />Options: `--props-file` (required)
+- **`utils loadClass`** Load a class.
+ <br />Options: `--class` (required)
+
+### Commits and the timeline
+
+- **`commit show_write_stats`** Show write stats of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showfiles`** Show file level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showpartitions`** Show partition level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commits compare`** Compare commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`commits show`** Show the commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`, `--includeArchivedTimeline` (`false`)
+- **`commits show_inflights`** Show inflight instants that are left longer than a certain duration.
+ <br />Options: `--lookbackInMins` (`0`)
+- **`commits showarchived`** Show the archived commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`commits sync`** Sync commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`diff file`** Check how file differs across range of commits.
+ <br />Options: `--fileId` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`diff partition`** Check how file differs across range of commits. It is meant to be used only for partitioned tables.
+ <br />Options: `--partitionPath` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`metadata timeline show active`** List all instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`metadata timeline show incomplete`** List all incomplete instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`show archived commit stats`** Read commits from archived files and show file group details.
+ <br />Options: `--archiveFolderPattern`, `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show archived commits`** Read commits from archived files and show details.
+ <br />Options: `--skipMetadata` (`true`), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`timeline show active`** List all instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--with-metadata-table` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`timeline show incomplete`** List all incomplete instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`trigger archival`** Trigger archival.
+ <br />Options: `--minCommits` (`20`), `--maxCommits` (`30`), `--commitsRetainedByCleaner` (`10`), `--enableMetadata` (`true`), `--sparkMemory` (`1G`), `--sparkMaster` (`local`)
+
+### Files, stats and log files
+
+- **`show fsview all`** Show entire file-system view.
+ <br />Options: `--pathRegex` (`*`), `--baseFileOnly` (`false`), `--maxInstant`, `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show fsview latest`** Show latest file-system view.
+ <br />Options: `--partitionPath`, `--baseFileOnly` (`false`), `--maxInstant`, `--merge` (`true`), `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile metadata`** Read commit metadata from log files.
+ <br />Options: `--logFilePathPattern` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile records`** Read records from log files.
+ <br />Options: `--limit` (`10`), `--logFilePathPattern` (required), `--mergeRecords` (`false`)
+- **`stats filesizes`** File Sizes. Display summary stats on sizes of files.
+ <br />Options: `--partitionPath` (`*/*/*`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`stats wa`** Write Amplification. Ratio of how many records were upserted to how many.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+
+### Table services
+
+- **`clean showpartitions`** Show partition level details of a clean.
+ <br />Options: `--clean` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`cleans run`** Run clean.
+ <br />Options: `--sparkMemory` (`4G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster`
+- **`cleans show`** Show the cleans.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--startTs`, `--endTs`, `--includeArchivedTimeline` (`false`), `--desc` (`false`), `--headeronly` (`false`)
+- **`clustering run`** Run Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--clusteringInstant`, `--propsFilePath`, `--hoodieConfigs`
+- **`clustering schedule`** Schedule Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`1g`), `--propsFilePath`, `--hoodieConfigs`
+- **`clustering scheduleAndExecute`** Run Clustering. Make a cluster plan first and execute that plan immediately.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`compaction repair`** Renames the files to make them consistent with the timeline as.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction run`** Run Compaction for given instant time.
+ <br />Options: `--parallelism` (`3`), `--schemaFilePath`, `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--compactionInstant`, `--propsFilePath`, `--hoodieConfigs`
+- **`compaction schedule`** Schedule Compaction.
+ <br />Options: `--sparkMemory` (`1G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster` (`local`)
+- **`compaction scheduleAndExecute`** Schedule compaction plan and execute this plan.
+ <br />Options: `--parallelism` (`3`), `--schemaFilePath`, `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`compaction show`** Shows compaction details for a specific compaction instant.
+ <br />Options: `--instant` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`compaction showarchived`** Shows compaction details for a specific compaction instant.
+ <br />Options: `--instant` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`compaction unschedule`** Unschedule Compaction.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--skipValidation` (`false`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction unscheduleFileId`** UnSchedule Compaction for a fileId.
+ <br />Options: `--fileId` (required), `--partitionPath`, `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--skipValidation` (`false`), `--dryRun` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compaction validate`** Validate Compaction.
+ <br />Options: `--instant` (required), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`2G`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compactions show all`** Shows all compactions that are in active timeline.
+ <br />Options: `--includeExtraMetadata` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`compactions showarchived`** Shows compaction details for specified time window.
+ <br />Options: `--includeExtraMetadata` (`false`), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`marker delete`** Delete the marker.
+ <br />Options: `--commit` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`1G`)
+
+### Rollback, savepoint, restore and repair
+
+- **`commit rollback`** Rollback a commit.
+ <br />Options: `--commit` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`), `--rollbackUsingMarkers` (`false`)
+- **`downgrade table`** Downgrades a table.
+ <br />Options: `--toVersion`, `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`rename partition`** Rename partition. Usage: rename partition --oldPartition <oldPartition> --newPartition <newPartition>.
+ <br />Options: `--oldPartition` (required), `--newPartition` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`repair addpartitionmeta`** Add partition metadata to a table, if not present.
+ <br />Options: `--dryrun` (`true`)
+- **`repair corrupted clean files`** Repair corrupted clean files.
+- **`repair deduplicate`** De-duplicate a partition path contains duplicates & produce repaired files to replace with.
+ <br />Options: `--duplicatedPartitionPath`, `--repairedOutputPath` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`), `--dryrun` (`true`), `--dedupeType` (`insert_type`)
+- **`repair deprecated partition`** Repair deprecated partition ("default"). Re-writes data from the deprecated partition into.
+ <br />Options: `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`repair migrate-partition-meta`** Migrate all partition meta file currently stored in text format.
+ <br />Options: `--dryrun` (`true`)
+- **`repair overwrite-hoodie-props`** Overwrite hoodie.properties with provided file. Risky operation. Proceed with caution!.
+ <br />Options: `--new-props-file` (required)
+- **`repair show empty commit metadata`** Show failed commits.
+- **`savepoint create`** Savepoint a commit.
+ <br />Options: `--commit` (required), `--user` (`default`), `--comments` (`default`), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`savepoint delete`** Delete the savepoint.
+ <br />Options: `--commit` (required), `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+- **`savepoint rollback`** Savepoint a commit.
+ <br />Options: `--savepoint` (required), `--sparkProperties`, `--sparkMaster`, `--lazyFailedWritesCleanPolicy` (`false`), `--sparkMemory` (`4G`)
+- **`savepoints show`** Show the savepoints.
+- **`show restore`** Show details of a restore instant.
+ <br />Options: `--instant` (required), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show restores`** List all restore instants.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeInflights` (`false`)
+- **`show rollback`** Show details of a rollback instant.
+ <br />Options: `--instant` (required), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show rollbacks`** List all rollback instants.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`upgrade table`** Upgrades a table.
+ <br />Options: `--toVersion`, `--sparkProperties`, `--sparkMaster`, `--sparkMemory` (`4G`)
+
+### Metadata table
+
+- **`metadata create`** Create the Metadata Table if it does not exist.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`)
+- **`metadata delete`** Remove the Metadata Table.
+ <br />Options: `--backup` (`true`)
+- **`metadata delete-record-index`** Delete the record index from Metadata Table.
+ <br />Options: `--backup` (`true`)
+- **`metadata init`** Update the metadata table from commits since the creation.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--readonly` (`false`)
+- **`metadata list-files`** Print a list of all files in a partition from the metadata.
+ <br />Options: `--partition`
+- **`metadata list-partitions`** List all partitions from metadata.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`)
+- **`metadata lookup-record-index`** Print Record index information for a record_key.
+ <br />Options: `--record_key` (required), `--partition_path` (required)
Review Comment:
Taken, with the description completed too. Confirmed `defaultValue = ""` at `MetadataCommand.java:399-400`, so `(required)` was wrong.
##########
website/docs/cli.md:
##########
@@ -253,29 +269,43 @@ hudi:trips->help
* create - Create a hoodie table if not present
* date - Displays the local date and time
* desc - Describe Hoodie Table properties
+* diff file - Check how file differs across range of commits
+* diff partition - Check how file differs across range of commits. It is meant to be used only for partitioned tables.
* downgrade table - Downgrades a table
* exit - Exits the shell
* export instants - Export Instants and their metadata from the Timeline
* fetch table schema - Fetches latest table schema
* hdfsparquetimport - Imports Parquet table to a hoodie table
Review Comment:
Dropped, along with the whole `help` transcript per your Spring Shell comment - so the listing that contradicted the reference is gone rather than patched.
##########
website/docs/cli.md:
##########
@@ -253,29 +269,43 @@ hudi:trips->help
* create - Create a hoodie table if not present
* date - Displays the local date and time
* desc - Describe Hoodie Table properties
+* diff file - Check how file differs across range of commits
+* diff partition - Check how file differs across range of commits. It is meant to be used only for partitioned tables.
* downgrade table - Downgrades a table
* exit - Exits the shell
* export instants - Export Instants and their metadata from the Timeline
* fetch table schema - Fetches latest table schema
* hdfsparquetimport - Imports Parquet table to a hoodie table
* help - List all commands usage
+* kerberos kdestroy - Destroy Kerberos authentication
Review Comment:
All four verified present on master and now listed. As above they live in the reference now, since the transcript block went; the listing entries are still there for the four you named.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(cli): document all 106 CLI commands [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19799:
URL: https://github.com/apache/hudi/pull/19799#discussion_r3946494543
##########
website/docs/cli.md:
##########
@@ -785,6 +762,442 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The comparison the CLI makes is `MetaFieldsMode#isWiderThan`, the same
+ helper the write path uses. Note the write path itself is stricter rather than equivalent: it rejects any writer
+ mode that differs from the table's, and consults `isWiderThan` only to choose the error wording.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits, and only commit, deltacommit, replacecommit and clustering actions. The
+timeline commands show every instant regardless of action and state, which is what you want when diagnosing a stuck
+table: a compaction sitting in `REQUESTED` never appears in `commits show`, and neither does a rollback, in any
+state.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | Annotate the Action column in both directions: `Rolls back ...` on a rollback instant, and `Rolled back by ...` on any instant a rollback targeted. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success. It only works on a table with a single partition field: the rewrite looks the partition field
+up by name in the dataframe schema, and on a multi-field table that name is the comma-joined list, so the job fails
+before writing anything.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (unset by default,
+which means `yarn` -- see the note under the Command reference legend) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
+
+Every command `hudi-cli` exposes, grouped by area, with its options and their defaults. An option marked
+`(required)` has no default and must be supplied; a value in backticks after an option is its default; an option
+listed with neither is optional and unset by default, which for most commands means the empty string reaches the
+command. The sections above cover the commonly used ones in more depth.
+
+One default is worth stating up front because it is not the empty string it looks like: an unset or blank
+`--sparkMaster` resolves to `yarn`. The CLI treats a blank value as absent and falls back to its own default, so
+every `--sparkMaster` shown below without a value runs against YARN unless you pass something. That differs from
+the compaction and marker commands, which show `local` explicitly.
Review Comment:
🤖 Small contradiction with the reference below: `marker delete` has `defaultValue = ""` for `--sparkMaster` (`MarkersCommand.java:41`), so it falls back to `yarn` like the others — and the entry at line 1101 correctly shows it without a value. Could this just say "the compaction commands and `trigger archival`, which show `local` explicitly"? Same sentence in `versioned_docs/version-1.2.0/cli.md:942`.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(cli): document all 106 CLI commands [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous merged PR #19799:
URL: https://github.com/apache/hudi/pull/19799
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(cli): document all 106 CLI commands [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on code in PR #19799:
URL: https://github.com/apache/hudi/pull/19799#discussion_r3946469667
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (empty by default) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
+
+Every command `hudi-cli` exposes, grouped by area, with its options and their defaults. An option marked
+`(required)` has no default and must be supplied; a value in backticks after an option is its default.
+The sections above cover the commonly used ones in more depth.
+
+Some entries are aliases of the same command rather than distinct ones. `refresh`, `metadata refresh`,
+`commits refresh`, `cleans refresh` and `savepoints refresh` are five names for one method that reloads the table
+metadata, and `temp query` / `temp_query`, `temp delete` / `temp_delete` and `temps show` / `temps_show` are
+underscore and space spellings of the same three commands.
+
+### Table and session
+
+- **`cleans refresh`** Refresh table metadata.
+- **`commits refresh`** Refresh table metadata.
+- **`connect`** Connect to a hoodie table.
+ <br />Options: `--path` (required), `--eventuallyConsistent` (`false`), `--initialCheckIntervalMs` (`2000`), `--maxWaitIntervalMs` (`300000`), `--maxCheckIntervalMs` (`7`), `--timeGeneratorType` (`WAIT_TO_ADJUST_SKEW`), `--maxExpectedClockSkewMs` (`200`), `--useDefaultLockProvider` (`false`)
+- **`create`** Create a hoodie table if not present.
+ <br />Options: `--path` (required), `--tableName` (required), `--tableType` (`COPY_ON_WRITE`), `--archiveLogFolder`, `--tableVersion`, `--payloadClass` (`org.apache.hudi.common.model.HoodieAvroPayload`)
+- **`desc`** Describe Hoodie Table properties.
+- **`fetch table schema`** Fetches latest table schema.
+ <br />Options: `--outputFilePath`
+- **`kerberos kdestroy`** Destroy Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`)
+- **`kerberos kinit`** Perform Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`), `--principal` (required), `--keytab` (required)
+- **`metadata refresh`** Refresh table metadata.
+- **`refresh`** Refresh table metadata.
+- **`savepoints refresh`** Refresh table metadata.
+- **`set`** Set spark launcher env to cli.
+ <br />Options: `--conf` (required)
+- **`show env`** Show spark launcher env by key.
+ <br />Options: `--key` (required)
+- **`show envs all`** Show spark launcher envs.
+- **`table change-table-type`** Change hudi table type to target type: COW or MOR.
+ <br />Options: `--target-type` (required), `--enable-compaction` (`true`), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`table delete-configs`** Delete the supplied table configs from the table.
+ <br />Options: `--comma-separated-configs` (required)
+- **`table recover-configs`** Recover table configs, from update/delete that failed midway.
+- **`table set-meta-fields-mode`** Set hoodie.meta.fields.mode on an existing table. This is the sanctioned way to change.
+ <br />Options: `--target-mode` (required), `--force` (`false`)
+- **`table update-configs`** Update the table configs with configs with provided file.
+ <br />Options: `--props-file` (required)
+- **`utils loadClass`** Load a class.
+ <br />Options: `--class` (required)
+
+### Commits and the timeline
+
+- **`commit show_write_stats`** Show write stats of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showfiles`** Show file level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showpartitions`** Show partition level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commits compare`** Compare commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`commits show`** Show the commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`, `--includeArchivedTimeline` (`false`)
+- **`commits show_inflights`** Show inflight instants that are left longer than a certain duration.
+ <br />Options: `--lookbackInMins` (`0`)
+- **`commits showarchived`** Show the archived commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`commits sync`** Sync commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`diff file`** Check how file differs across range of commits.
+ <br />Options: `--fileId` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`diff partition`** Check how file differs across range of commits. It is meant to be used only for partitioned tables.
+ <br />Options: `--partitionPath` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`metadata timeline show active`** List all instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`metadata timeline show incomplete`** List all incomplete instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`show archived commit stats`** Read commits from archived files and show file group details.
+ <br />Options: `--archiveFolderPattern`, `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show archived commits`** Read commits from archived files and show details.
+ <br />Options: `--skipMetadata` (`true`), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`timeline show active`** List all instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--with-metadata-table` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`timeline show incomplete`** List all incomplete instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`trigger archival`** Trigger archival.
+ <br />Options: `--minCommits` (`20`), `--maxCommits` (`30`), `--commitsRetainedByCleaner` (`10`), `--enableMetadata` (`true`), `--sparkMemory` (`1G`), `--sparkMaster` (`local`)
+
+### Files, stats and log files
+
+- **`show fsview all`** Show entire file-system view.
+ <br />Options: `--pathRegex` (`*`), `--baseFileOnly` (`false`), `--maxInstant`, `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show fsview latest`** Show latest file-system view.
+ <br />Options: `--partitionPath`, `--baseFileOnly` (`false`), `--maxInstant`, `--merge` (`true`), `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile metadata`** Read commit metadata from log files.
+ <br />Options: `--logFilePathPattern` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile records`** Read records from log files.
+ <br />Options: `--limit` (`10`), `--logFilePathPattern` (required), `--mergeRecords` (`false`)
+- **`stats filesizes`** File Sizes. Display summary stats on sizes of files.
+ <br />Options: `--partitionPath` (`*/*/*`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`stats wa`** Write Amplification. Ratio of how many records were upserted to how many.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+
+### Table services
+
+- **`clean showpartitions`** Show partition level details of a clean.
+ <br />Options: `--clean` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`cleans run`** Run clean.
+ <br />Options: `--sparkMemory` (`4G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster`
+- **`cleans show`** Show the cleans.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--startTs`, `--endTs`, `--includeArchivedTimeline` (`false`), `--desc` (`false`), `--headeronly` (`false`)
+- **`clustering run`** Run Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--clusteringInstant`, `--propsFilePath`, `--hoodieConfigs`
Review Comment:
Fixed, all six now read `yarn` rather than the raw constant. Also added the wider note voonhous raised below, since an unset `--sparkMaster` resolves to `yarn` too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(cli): document all 106 CLI commands [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on code in PR #19799:
URL: https://github.com/apache/hudi/pull/19799#discussion_r3946470982
##########
website/docs/cli.md:
##########
@@ -160,6 +160,22 @@ Kerberos authentication success
If you see "Kerberos authentication success" in the command output, it means Kerberos authentication has been successful.
+**kerberos kdestroy** is the counterpart: it logs the current user out of the keytab and resets the cached
+`UserGroupInformation`, which is what you want before authenticating as a different principal in the same session.
+
+```shell
+hudi->kerberos kdestroy --krb5conf /etc/krb5.conf
Review Comment:
Both switched to `java`, so all sixteen new transcript blocks match.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(cli): document all 106 CLI commands [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on PR #19799:
URL: https://github.com/apache/hudi/pull/19799#issuecomment-5474558307
Hi @voonhous I have update the PR description and documentation changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(cli): document all 106 CLI commands [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19799:
URL: https://github.com/apache/hudi/pull/19799#discussion_r3900947906
##########
website/docs/cli.md:
##########
@@ -785,6 +873,431 @@ table change-table-type COW
╚════════════════════════════════════════════════╧══════════════════════════════════════╧══════════════════════════════════════╝
```
+### Changing the Meta Fields Mode
+
+`hoodie.meta.fields.mode` decides which of Hudi's meta columns are physically written into base files. It is a
+storage-layout decision baked in at write time, so a write can never change it on an existing table.
+`table set-meta-fields-mode` is the sanctioned way to change it.
+
+```java
+hudi:trips->table set-meta-fields-mode --target-mode COMMIT_TIME_ONLY
+```
+
+`--target-mode` accepts `ALL`, `NONE`, `COMMIT_TIME_ONLY`, `FILE_NAME_ONLY` or `COMMIT_TIME_AND_FILE_NAME`. The value
+is resolved case-insensitively and trimmed, so `commit_time_only` is accepted too. Setting the mode the table is
+already in is a no-op and reports as much.
+
+On a table that already has commits, two guards apply, because this command changes the table property without
+rewriting a single existing file:
+
+- **Widening is refused outright**, and `--force` does not override it. Widening means the target mode populates a
+ meta column the current mode does not. Since earlier files are not rewritten, the table would advertise a column
+ that is null for every row written so far, and incremental queries and file-name lookups silently skip exactly
+ those rows. To widen, recreate the table. The CLI uses the same predicate as the write path
+ (`BaseHoodieWriteClient#validateAgainstTableProperties`), so the two cannot disagree about which transitions are
+ legal.
+- **Narrowing needs `--force`** (default `false`). It leaves mixed-mode files: old commits keep the old layout, new
+ commits use the new one, and incremental and file-pruning semantics differ between the two sets. Passing `--force`
+ logs a warning recording the transition and the commit count.
+
+Neither guard applies to a table with no commits, where the mode can be set freely.
+
+### Inspecting the Timeline
+
+`commits show` lists completed commits. The timeline commands show every instant regardless of action and state, which
+is what you want when diagnosing a stuck table: a compaction sitting in `REQUESTED`, or a rollback that never
+completed, never appears in `commits show`.
+
+```java
+hudi:trips->timeline show active --limit 10
+hudi:trips->timeline show incomplete
+```
+
+Both print `Instant`, `Action`, `State`, and the `Requested` / `Inflight` / `Completed` file modification times.
+`timeline show incomplete` restricts the listing to instants that are not yet completed.
+
+| Option | Default | Applies to | Description |
+| --- | --- | --- | --- |
+| `--limit` | `10` | both | Number of rows to display. |
+| `--sortBy` | unset | both | Field to sort by. |
+| `--desc` | `false` | both | Reverse the ordering. |
+| `--headeronly` | `false` | both | Print the header only. |
+| `--show-rollback-info` | `false` | both | For rollback instants, also show the instant being rolled back. |
+| `--show-time-seconds` | `false` | both | Include seconds in the instant file modification times. |
+| `--with-metadata-table` | `false` | `timeline show active` only | Show the metadata table timeline alongside the data table, adding `MT Action`, `MT State` and the three matching MT time columns. |
+
+The metadata table has its own timeline, and the two can disagree when a metadata commit fails. To read it directly:
+
+```java
+hudi:trips->metadata timeline show active --limit 10
+hudi:trips->metadata timeline show incomplete
+```
+
+These accept `--limit`, `--sortBy`, `--desc`, `--headeronly` and `--show-time-seconds`. They have no
+`--show-rollback-info` and no `--with-metadata-table`, since they are already scoped to the metadata table.
+
+### Diffing a File or Partition
+
+`diff file` and `diff partition` replay the timeline and show every commit that touched a given file group or
+partition, which is the quickest way to answer "what has been writing to this file". Both report the standard commit
+columns plus the write statistics for the matching entries only.
+
+```java
+hudi:trips->diff file --fileId 5f8a1e0b-1b4b-4a3f-9b1a-2c7d6e5f4a3b-0 --limit 10
+hudi:trips->diff partition --partitionPath 2026/08/26 --includeArchivedTimeline true
+```
+
+`diff file` takes `--fileId` and `diff partition` takes `--partitionPath` as a partition path relative to the table
+base path. `diff partition` is only meaningful on a partitioned table. Both then share these options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `--includeArchivedTimeline` | `false` | Also scan archived instants, not just the active timeline. |
+| `--startTs` | unset, meaning now minus 10 days | Start of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--endTs` | unset, meaning now minus 1 day | End of the instant range. Only applied when `--includeArchivedTimeline` is `true`. |
+| `--limit` | `-1`, meaning no limit | Number of rows to display. |
+| `--sortBy` | unset | Field to sort by. |
+| `--desc` | `false` | Reverse the ordering. |
+| `--headeronly` | `false` | Print the header only. |
+
+Note the interaction between the three range options, which is easy to get wrong. `--startTs` and `--endTs` are used
+to select archived instants only. With the default `--includeArchivedTimeline false` the whole active timeline is
+scanned and both bounds are ignored, so passing a narrow range does not restrict the output. Set
+`--includeArchivedTimeline true` for the bounds to take effect, and note that the archived range is then merged with
+the full active timeline rather than replacing it.
+
+### Repairing a Table
+
+`repair show empty commit metadata` scans completed instants on the active timeline and reports the ones whose
+metadata file is empty, which is what a commit interrupted between file creation and metadata write leaves behind.
+
+```java
+hudi:trips->repair show empty commit metadata
+```
+
+Note that this command writes its findings to the CLI log at `WARN` level rather than returning a table, so with the
+default logging configuration you will see the `Empty Commit: ...` lines in the console log rather than in a rendered
+result. It only reports; it does not modify the timeline.
+
+`rename partition` rewrites the data under one partition value into another, as a Spark job, and deletes the old
+partition on success.
+
+```java
+hudi:trips->set --conf SPARK_HOME=<SPARK_HOME>
+hudi:trips->rename partition --oldPartition 2026/08/26 --newPartition 2026-08-26 --sparkMaster local[2]
+```
+
+`repair deprecated partition` is the special case of that rename for tables written before Hudi settled on its
+placeholder for the null partition value: it rewrites data from the deprecated `default` partition into
+`__HIVE_DEFAULT_PARTITION__`.
+
+```java
+hudi:trips->repair deprecated partition --sparkMaster local[2]
+```
+
+Both take `--sparkProperties` (a Spark properties file path, empty by default), `--sparkMaster` (empty by default) and
+`--sparkMemory` (`4G` by default). Both read the old partition, rewrite those records under the new partition value,
+and then issue a `delete_partition` write against the old one, so the change goes through the timeline rather than
+behind it. Both are a no-op when the old partition holds no records.
+
+They differ in one respect worth knowing: `rename partition` additionally removes the old partition directory from
+storage after the delete write, logging a warning if that removal fails, whereas `repair deprecated partition` leaves
+the emptied `default` directory in place. Either way these rewrite data, so take a savepoint first if the table
+matters.
+
+### Auditing Storage Locks
+
+When a table uses a storage-based lock provider, the lock provider can record every lock transition to a set of JSONL
+files, so that a suspected concurrency violation can be reconstructed after the fact. The audit is off by default and
+is controlled by a config file next to the locks themselves, at
+`<basePath>/.hoodie/.locks/audit_enabled.json`; the audit records land in `<basePath>/.hoodie/.locks/audit/`.
+
+```java
+hudi:trips->locks audit enable
+Lock audit enabled successfully.
+Audit config written to: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files will be stored at: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit status` reports whether auditing is on, and where both the config and the records live. A table that has
+never had auditing enabled reports `DISABLED` with the config file marked `(not found)`.
+
+```java
+hudi:trips->locks audit status
+Lock Audit Status: ENABLED
+Table: /user/hive/warehouse/table1
+Config file: /user/hive/warehouse/table1/.hoodie/.locks/audit_enabled.json
+Audit files location: /user/hive/warehouse/table1/.hoodie/.locks/audit
+```
+
+`locks audit validate` is the reason to collect the records. It parses every `.jsonl` file in the audit folder into
+transaction windows and checks them against each other. Overlapping windows are reported as errors, since two writers
+holding the lock at once is exactly the violation the lock provider exists to prevent. A transaction that never
+released its lock is reported as a warning, which usually means a driver OOM or a non-graceful shutdown rather than a
+correctness problem. The verdict is `PASSED`, `WARNING` when only warnings were found, or `FAILED` when any error was.
+
+```java
+hudi:trips->locks audit validate
+Validation Result: PASSED
+Audit Files: 12 total, 12 parsed successfully, 0 failed to parse
+Transactions Validated: 12
+Issues Found: 0
+Details: All audit lock transactions validated successfully
+```
+
+With no audit folder or no audit files the command reports `PASSED` with zero transactions validated, so a `PASSED`
+verdict on its own does not prove that auditing was ever on. Check `locks audit status` first.
+
+`locks audit cleanup` prunes old records. `--ageDays` defaults to `7` and `--dryRun` defaults to `false`, so run it
+with `--dryRun true` first to see what it would remove.
+
+```java
+hudi:trips->locks audit cleanup --dryRun true --ageDays 30
+```
+
+`locks audit disable` turns auditing off. It keeps the existing records by default; pass `--keepAuditFiles false` to
+delete them at the same time, which internally runs the same cleanup with no age threshold.
+
+```java
+hudi:trips->locks audit disable --keepAuditFiles true
+```
+
+All five commands require a table to be connected, and report `No Hudi table loaded. Please connect to a table first.`
+otherwise.
+
+## Command reference
+
+Every command `hudi-cli` exposes, grouped by area, with its options and their defaults. An option marked
+`(required)` has no default and must be supplied; a value in backticks after an option is its default.
+The sections above cover the commonly used ones in more depth.
+
+Some entries are aliases of the same command rather than distinct ones. `refresh`, `metadata refresh`,
+`commits refresh`, `cleans refresh` and `savepoints refresh` are five names for one method that reloads the table
+metadata, and `temp query` / `temp_query`, `temp delete` / `temp_delete` and `temps show` / `temps_show` are
+underscore and space spellings of the same three commands.
+
+### Table and session
+
+- **`cleans refresh`** Refresh table metadata.
+- **`commits refresh`** Refresh table metadata.
+- **`connect`** Connect to a hoodie table.
+ <br />Options: `--path` (required), `--eventuallyConsistent` (`false`), `--initialCheckIntervalMs` (`2000`), `--maxWaitIntervalMs` (`300000`), `--maxCheckIntervalMs` (`7`), `--timeGeneratorType` (`WAIT_TO_ADJUST_SKEW`), `--maxExpectedClockSkewMs` (`200`), `--useDefaultLockProvider` (`false`)
+- **`create`** Create a hoodie table if not present.
+ <br />Options: `--path` (required), `--tableName` (required), `--tableType` (`COPY_ON_WRITE`), `--archiveLogFolder`, `--tableVersion`, `--payloadClass` (`org.apache.hudi.common.model.HoodieAvroPayload`)
+- **`desc`** Describe Hoodie Table properties.
+- **`fetch table schema`** Fetches latest table schema.
+ <br />Options: `--outputFilePath`
+- **`kerberos kdestroy`** Destroy Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`)
+- **`kerberos kinit`** Perform Kerberos authentication.
+ <br />Options: `--krb5conf` (`/etc/krb5.conf`), `--principal` (required), `--keytab` (required)
+- **`metadata refresh`** Refresh table metadata.
+- **`refresh`** Refresh table metadata.
+- **`savepoints refresh`** Refresh table metadata.
+- **`set`** Set spark launcher env to cli.
+ <br />Options: `--conf` (required)
+- **`show env`** Show spark launcher env by key.
+ <br />Options: `--key` (required)
+- **`show envs all`** Show spark launcher envs.
+- **`table change-table-type`** Change hudi table type to target type: COW or MOR.
+ <br />Options: `--target-type` (required), `--enable-compaction` (`true`), `--parallelism` (`3`), `--sparkMaster` (`local`), `--sparkMemory` (`4G`), `--retry` (`1`), `--propsFilePath`, `--hoodieConfigs`
+- **`table delete-configs`** Delete the supplied table configs from the table.
+ <br />Options: `--comma-separated-configs` (required)
+- **`table recover-configs`** Recover table configs, from update/delete that failed midway.
+- **`table set-meta-fields-mode`** Set hoodie.meta.fields.mode on an existing table. This is the sanctioned way to change.
+ <br />Options: `--target-mode` (required), `--force` (`false`)
+- **`table update-configs`** Update the table configs with configs with provided file.
+ <br />Options: `--props-file` (required)
+- **`utils loadClass`** Load a class.
+ <br />Options: `--class` (required)
+
+### Commits and the timeline
+
+- **`commit show_write_stats`** Show write stats of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showfiles`** Show file level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commit showpartitions`** Show partition level details of a commit.
+ <br />Options: `--createView`, `--commit` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`commits compare`** Compare commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`commits show`** Show the commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`, `--includeArchivedTimeline` (`false`)
+- **`commits show_inflights`** Show inflight instants that are left longer than a certain duration.
+ <br />Options: `--lookbackInMins` (`0`)
+- **`commits showarchived`** Show the archived commits.
+ <br />Options: `--includeExtraMetadata` (`false`), `--createView`, `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--partition`
+- **`commits sync`** Sync commits with another Hoodie table.
+ <br />Options: `--path` (required)
+- **`diff file`** Check how file differs across range of commits.
+ <br />Options: `--fileId` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`diff partition`** Check how file differs across range of commits. It is meant to be used only for partitioned tables.
+ <br />Options: `--partitionPath` (required), `--startTs`, `--endTs`, `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--includeArchivedTimeline` (`false`)
+- **`metadata timeline show active`** List all instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`metadata timeline show incomplete`** List all incomplete instants in active timeline of metadata table.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-time-seconds` (`false`)
+- **`show archived commit stats`** Read commits from archived files and show file group details.
+ <br />Options: `--archiveFolderPattern`, `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show archived commits`** Read commits from archived files and show details.
+ <br />Options: `--skipMetadata` (`true`), `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`timeline show active`** List all instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--with-metadata-table` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`timeline show incomplete`** List all incomplete instants in active timeline.
+ <br />Options: `--limit` (`10`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`), `--show-rollback-info` (`false`), `--show-time-seconds` (`false`)
+- **`trigger archival`** Trigger archival.
+ <br />Options: `--minCommits` (`20`), `--maxCommits` (`30`), `--commitsRetainedByCleaner` (`10`), `--enableMetadata` (`true`), `--sparkMemory` (`1G`), `--sparkMaster` (`local`)
+
+### Files, stats and log files
+
+- **`show fsview all`** Show entire file-system view.
+ <br />Options: `--pathRegex` (`*`), `--baseFileOnly` (`false`), `--maxInstant`, `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show fsview latest`** Show latest file-system view.
+ <br />Options: `--partitionPath`, `--baseFileOnly` (`false`), `--maxInstant`, `--merge` (`true`), `--includeMax` (`false`), `--includeInflight` (`false`), `--excludeCompaction` (`false`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile metadata`** Read commit metadata from log files.
+ <br />Options: `--logFilePathPattern` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`show logfile records`** Read records from log files.
+ <br />Options: `--limit` (`10`), `--logFilePathPattern` (required), `--mergeRecords` (`false`)
+- **`stats filesizes`** File Sizes. Display summary stats on sizes of files.
+ <br />Options: `--partitionPath` (`*/*/*`), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`stats wa`** Write Amplification. Ratio of how many records were upserted to how many.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+
+### Table services
+
+- **`clean showpartitions`** Show partition level details of a clean.
+ <br />Options: `--clean` (required), `--limit` (`-1`), `--sortBy`, `--desc` (`false`), `--headeronly` (`false`)
+- **`cleans run`** Run clean.
+ <br />Options: `--sparkMemory` (`4G`), `--propsFilePath`, `--hoodieConfigs`, `--sparkMaster`
+- **`cleans show`** Show the cleans.
+ <br />Options: `--limit` (`-1`), `--sortBy`, `--startTs`, `--endTs`, `--includeArchivedTimeline` (`false`), `--desc` (`false`), `--headeronly` (`false`)
+- **`clustering run`** Run Clustering.
+ <br />Options: `--sparkMaster` (`SparkUtil.DEFAULT_SPARK_MASTER`), `--sparkMemory` (`4g`), `--parallelism` (`1`), `--retry` (`1`), `--clusteringInstant`, `--propsFilePath`, `--hoodieConfigs`
Review Comment:
🤖 This entry lists the `--sparkMaster` default as the raw Java constant `SparkUtil.DEFAULT_SPARK_MASTER` rather than its value. That constant resolves to `yarn` (`SparkUtil.DEFAULT_SPARK_MASTER = "yarn"`), so it would help to substitute the concrete value here — the way the other Spark commands in this reference already do (e.g. compaction/marker show `local`). The same unresolved constant appears on the `clustering schedule` / `clustering scheduleAndExecute` entries (lines 1177, 1179) and on `metadata create` / `metadata init` / `metadata list-partitions` (lines 1246, 1252, 1256). It might also be worth a one-line note that this default is `yarn`, since that differs from the `local` default used by the compaction and other Spark commands and could surprise a reader running these on a non-YARN setup.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org