[PR] [docs] Reorganize Flink guides and add SVG diagrams [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi opened a new pull request, #9725:
URL: https://github.com/apache/paimon/pull/9725

   ### Purpose
   
   The Flink documentation mixes setup, everyday SQL usage, and operational details in a flat navigation, while the procedure reference is a single large HTML table. Reorganize the guides by task and make the examples and operational constraints easier to follow.
   
   - Add an overview, dedicated installation and runtime configuration guides, and a troubleshooting guide while preserving existing page URLs.
   - Split all 54 procedures into eight topic pages with readable parameter lists and SQL examples.
   - Refine the quick start, DDL, read/write, lookup, action, consumer, and savepoint guidance against the implementation.
   - Add seven accessible SVG diagrams covering the read/write flow, query modes, lookup distribution, incremental boundaries, overwrite scope, consumer progress, and savepoint recovery.
   
   ### Tests
   
   - Docusaurus production build passed after rebasing onto the latest upstream master.
   - Validated local links, anchors, and assets in the generated Flink pages, all 54 procedure names, and XML/accessibility structure for all seven SVGs.
   - Visually checked the generated guides and diagrams; checked the merge shell examples with `bash -n`.
   - `git diff --check` passed.
   
   Documentation-only change; SQL examples were checked against the implementation, but no Flink jobs were executed.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [docs] Reorganize Flink guides and add SVG diagrams [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #9725:
URL: https://github.com/apache/paimon/pull/9725


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [docs] Reorganize Flink guides and add SVG diagrams [paimon]

["leaves12138 (via GitHub)" <gi...@apache.org>]

leaves12138 commented on code in PR #9725:
URL: https://github.com/apache/paimon/pull/9725#discussion_r3980137121


##########
docs/docs/flink/procedures/repair.md:
##########
@@ -0,0 +1,206 @@
+---
+title: "Cleanup and Repair"
+sidebar_position: 7
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Cleanup and Repair
+
+Inspect orphan files or repair missing files, manifests, and table metadata.
+
+See [Procedures](../procedures) for Flink version requirements, argument conventions, and catalog selection.
+
+## remove_orphan_files
+
+To remove the orphan data files and metadata files. Arguments:
+
+- `table`: the target table identifier. Cannot be empty, you can use database_name.* to clean whole database.
+
+- `olderThan`: to avoid deleting newly written files, this procedure only deletes orphan files older than 1 day by default. This argument can modify the interval.
+
+- `dryRun`: when true, view only orphan files, don't actually remove files. Default is false.
+
+- `parallelism`: The maximum number of concurrent deleting files. By default is the number of processors available to the Java virtual machine.
+
+- `mode`: The mode of remove orphan clean procedure (local or distributed) . By default is distributed.
+
+**Syntax**
+
+```sql
+-- Use named argument
+CALL [catalog.]sys.remove_orphan_files(
+    `table` => 'identifier',
+    older_than => 'olderThan',
+    dry_run => 'dryRun',
+    mode => 'mode'
+);
+
+-- Use indexed argument
+CALL [catalog.]sys.remove_orphan_files('identifier');
+
+CALL [catalog.]sys.remove_orphan_files('identifier', 'olderThan');
+
+CALL [catalog.]sys.remove_orphan_files('identifier', 'olderThan', 'dryRun');
+
+CALL [catalog.]sys.remove_orphan_files('identifier', 'olderThan', 'dryRun','parallelism');
+
+CALL [catalog.]sys.remove_orphan_files('identifier', 'olderThan', 'dryRun','parallelism','mode');
+```
+
+**Example**
+
+```sql
+CALL sys.remove_orphan_files(`table` => 'default.T', older_than => '2023-10-31 12:00:00');
+
+CALL sys.remove_orphan_files(`table` => 'default.*', older_than => '2023-10-31 12:00:00');
+
+CALL sys.remove_orphan_files(`table` => 'default.T', older_than => '2023-10-31 12:00:00', dry_run => true);
+
+CALL sys.remove_orphan_files(
+    `table` => 'default.T',
+    older_than => '2023-10-31 12:00:00',
+    dry_run => false,
+    parallelism => 5
+);
+
+CALL sys.remove_orphan_files(
+    `table` => 'default.T',
+    older_than => '2023-10-31 12:00:00',
+    dry_run => false,
+    parallelism => 5,
+    mode => 'local'
+);
+```
+
+## remove_unexisting_files
+
+Procedure to remove unexisting data files from manifest entries. See [Java docs](https://paimon.apache.org/docs/master/api/java/org/apache/paimon/flink/action/RemoveUnexistingFilesAction.html) for detailed use cases. Arguments:
+
+- `table`: the target table identifier. Cannot be empty, you can use database_name.* to clean whole database.
+
+- `dry_run` (optional): only check what files will be removed, but not really remove them. Default is false.
+
+- `parallelism` (optional): number of parallelisms to check files in the manifests.
+
+Note that user is on his own risk using this procedure, which may cause data loss when used outside from the use cases listed in Java docs.
+
+**Syntax**
+
+```sql
+-- Use named argument
+CALL [catalog.]sys.remove_unexisting_files(
+    `table` => 'identifier',
+    dry_run => 'dryRun',
+    parallelism => parallelism
+);
+
+-- Use indexed argument
+CALL [catalog.]sys.remove_unexisting_files('identifier');
+
+CALL [catalog.]sys.remove_unexisting_files('identifier', 'dryRun', 'parallelism');
+```
+
+**Example**
+
+```sql
+-- remove unexisting data files in the table `mydb.myt`
+CALL sys.remove_unexisting_files(`table` => 'mydb.myt');
+
+-- only check what files will be removed, but not really remove them (dry run)
+CALL sys.remove_unexisting_files(`table` => 'mydb.myt', `dry_run` = true);
+```
+
+## remove_unexisting_manifests
+
+Procedure to remove unexisting manifest file from manifset-list. for detailed use cases. Arguments:
+
+- `table`: the target table identifier. Cannot be empty, you can use database.table$branch_xx to remove branch table unexisting manifest file.
+
+Note that user is on his own risk using this procedure, which may cause data loss when used outside from the use cases listed in Java docs.
+
+**Syntax**
+
+```sql
+-- Use named argument
+CALL [catalog.]sys.remove_unexisting_files(`table` => 'identifier');
+```
+
+**Example**
+
+```sql
+-- remove unexisting manifest file in the table `mydb.myt`
+CALL sys.remove_unexisting_manifests(`table` => 'mydb.myt');

Review Comment:
   Non-blocking, carried over from the old procedure table: please correct the manifest-repair reference in a follow-up. Its Syntax block currently invokes `remove_unexisting_files` instead of `remove_unexisting_manifests`, and the manifest procedure's actual named argument is `tableId`, not `table`. On Flink 1.20.1 I verified that the named call shown here fails validation, while `CALL sys.remove_unexisting_manifests(tableId => 'default.repair_target')` succeeds and preserves the table's rows. Please update the Syntax block and both manifest-repair examples consistently; a positional call also avoids the parameter-name mismatch. This is not a regression introduced by this reorganization.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org