You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@paimon.apache.org by lz...@apache.org on 2024/06/26 02:39:27 UTC
(paimon) branch master updated: [docs] Document branch (#3544)
This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/paimon.git
The following commit(s) were added to refs/heads/master by this push:
new 6a79270c4 [docs] Document branch (#3544)
6a79270c4 is described below
commit 6a79270c4fd5a9bd868de91465770e475f5ebf90
Author: GuojunLi <li...@bytedance.com>
AuthorDate: Wed Jun 26 10:39:22 2024 +0800
[docs] Document branch (#3544)
---
docs/content/flink/procedures.md | 64 ++++++++++++++
docs/content/maintenance/manage-branches.md | 131 ++++++++++++++++++++++++++++
2 files changed, 195 insertions(+)
diff --git a/docs/content/flink/procedures.md b/docs/content/flink/procedures.md
index f6d50cf37..411f6a58e 100644
--- a/docs/content/flink/procedures.md
+++ b/docs/content/flink/procedures.md
@@ -295,6 +295,70 @@ All available procedures are listed below.
-- repair all tables in a specific partition<br/>
CALL sys.rewrite_file_index('test_db.T', 'pt=a')<br/><br/>
</td>
+ <tr>
+ <td>create_branch</td>
+ <td>
+ -- based on the specified snapshot <br/>
+ CALL [catalog.]sys.create_branch('identifier', 'branchName', snapshotId) <br/>
+ -- based on the specified tag <br/>
+ CALL [catalog.]sys.create_branch('identifier', 'branchName', 'tagName')
+ -- create empty branch <br/>
+ CALL [catalog.]sys.create_branch('identifier', 'branchName')
+ </td>
+ <td>
+ To create a branch based on given snapshot / tag, or just create empty branch. Arguments:
+ <li>identifier: the target table identifier. Cannot be empty.</li>
+ <li>branchName: name of the new branch.</li>
+ <li>snapshotId (Long): id of the snapshot which the new branch is based on.</li>
+ <li>tagName: name of the tag which the new branch is based on.</li>
+ </td>
+ <td>
+ CALL sys.create_branch('default.T', 'branch1', 10)<br/><br/>
+ CALL sys.create_branch('default.T', 'branch1', 'tag1')<br/><br/>
+ CALL sys.create_branch('default.T', 'branch1')<br/><br/>
+ </td>
+ </tr>
+ <tr>
+ <td>delete_branch</td>
+ <td>
+ CALL [catalog.]sys.delete_branch('identifier', 'branchName')
+ </td>
+ <td>
+ To delete a branch. Arguments:
+ <li>identifier: the target table identifier. Cannot be empty.</li>
+ <li>branchName: name of the branch to be deleted.</li>
+ </td>
+ <td>
+ CALL sys.delete_branch('default.T', 'branch1')
+ </td>
+ </tr>
+ <tr>
+ <td>merge_branch</td>
+ <td>
+ CALL [catalog.]sys.merge_branch('identifier', 'branchName')
+ </td>
+ <td>
+ To merge a branch to main branch. Arguments:
+ <li>identifier: the target table identifier. Cannot be empty.</li>
+ <li>branchName: name of the branch to be merged.</li>
+ </td>
+ <td>
+ CALL sys.merge_branch('default.T', 'branch1')
+ </td>
+ </tr>
+ <tr>
+ <td>replace_branch</td>
+ <td>
+ CALL [catalog.]sys.replace_branch('identifier', 'branchName')
+ </td>
+ <td>
+ To replace main branch with specified branch. Arguments:
+ <li>identifier: the target table identifier. Cannot be empty.</li>
+ <li>branchName: name of the branch to be replaced.</li>
+ </td>
+ <td>
+ CALL sys.replace_branch('default.T', 'branch1')
+ </td>
</tr>
</tbody>
</table>
diff --git a/docs/content/maintenance/manage-branches.md b/docs/content/maintenance/manage-branches.md
new file mode 100644
index 000000000..b9671153c
--- /dev/null
+++ b/docs/content/maintenance/manage-branches.md
@@ -0,0 +1,131 @@
+---
+title: "Manage Branches"
+weight: 11
+type: docs
+aliases:
+- /maintenance/manage-branches.html
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Manage Branches
+
+In streaming data processing, it's difficult to correct data for it may affect the existing data, and users will see the streaming provisional results, which is not expected.
+
+We suppose the branch that the existing workflow is processing on is 'main' branch, by creating custom data branch, it can help to do experimental tests and data validating for the new job on the existing table, which doesn't need to stop the existing reading / writing workflows and no need to copy data from the main branch.
+
+By merge or replace branch operations, users can complete the correcting of data.
+
+## Create Branches
+
+Paimon supports creating branch from a specific tag or snapshot, or just creating an empty branch which means the initial state of the created branch is like an empty table.
+
+{{< tabs "create-branches" >}}
+
+{{< tab "Flink" >}}
+
+Run the following sql:
+
+```sql
+-- create branch named 'branch1' from tag 'tag1'
+CALL sys.create_branch('default.T', 'branch1', 'tag1');
+
+-- create branch named 'branch1' from snapshot 1
+CALL sys.create_branch('default.T', 'branch1', 1);
+
+-- create empty branch named 'branch1'
+CALL sys.create_branch('default.T', 'branch1');
+```
+{{< /tab >}}
+
+{{< /tabs >}}
+
+## Delete Branches
+
+You can delete branch by its name.
+
+{{< tabs "delete-branches" >}}
+
+{{< tab "Flink" >}}
+
+Run the following sql:
+
+```sql
+CALL sys.delete_branch('default.T', 'branch1');
+```
+{{< /tab >}}
+
+{{< /tabs >}}
+
+## Read / Write With Branch
+
+You can read or write with branch as below.
+
+{{< tabs "read-write-branch" >}}
+
+{{< tab "Flink" >}}
+
+```sql
+-- read from branch 'branch1'
+SELECT * FROM t /*+ OPTIONS('branch' = 'branch1') */;
+
+-- write to branch 'branch1'
+INSERT INTO t /*+ OPTIONS('branch' = 'branch1') */ SELECT ...
+```
+
+{{< /tab >}}
+
+{{< /tabs >}}
+
+## Merge Branch
+
+Merging the custom branch to main will delete all the snapshots, tags and schemas in the main branch that are created after the branch's initial tag. And copy snapshots, tags and schemas from the branch to the main branch.
+
+{{< tabs "merge-branch" >}}
+
+{{< tab "Flink" >}}
+
+```sql
+CALL sys.merge_branch('default.T', 'branch1');
+```
+
+{{< /tab >}}
+
+{{< /tabs >}}
+
+## Replace Branch
+
+Replacing main branch with custom branch will copy the snapshots, tags and schemas which should be copied from the main branch to target branch, and then the custom branch will be the new 'main' branch.
+
+Suppose there is a job read and write the specified branch, by replacing main branch with the created branch, we don't need to do
+anything with this job, for the new branch is totally complete now, and the 'main' branch is pointed to the new branch now.
+
+We cannot merge branch to main here because the new job will still read and write the specified branch which will be completely independent of main.
+
+{{< tabs "replace-branch" >}}
+
+{{< tab "Flink" >}}
+
+```sql
+CALL sys.replace_branch('default.T', 'branch1');
+```
+
+{{< /tab >}}
+
+{{< /tabs >}}
\ No newline at end of file