You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "xuzifu666 (via GitHub)" <gi...@apache.org> on 2026/04/27 08:00:00 UTC

[PR] feat(datafusion): support branch_manager and system table [paimon-rust]

xuzifu666 opened a new pull request, #290:
URL: https://github.com/apache/paimon-rust/pull/290

   <!--
   Thank you very much for contributing to Paimon Rust - we are happy that you want to help us improve it. To help the community review your contribution in the best possible way, please go through the checklist below, which will get the contribution into a shape in which it can be best reviewed.
   
   ## Contribution Checklist
   
     - Make sure that the pull request corresponds to a [GitHub issue](https://github.com/apache/paimon-rust/issues). Exceptions are made for typos in documentation or comments, which need no issue.
   
     - Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
   
     - Make sure that the change passes the automated tests, i.e., `cargo test` passes.
   
     - Each pull request should address only one issue, not mix up code from multiple issues.
   
   **(The sections below can be removed for hotfixes or typos)**
   -->
   
   ### Purpose
   
   <!-- Linking this pull request to the issue -->
   Linked issue: close #xxx
   
   Branch management capabilities are frequently used in feature-driven scenarios, but are not currently supported in the Rust version. 
   This PR adds support for branch_manager and branch system table capabilities in the datafusion module.
   
   <!-- What is the purpose of the change -->
   
   ### Brief change log
   
   <!-- Please describe the changes made in this pull request and explain how they address the issue -->
   
   ### Tests
   
   <!-- List unit tests or integration cases to verify this change -->
   
   ### API and Format
   
   <!-- Does this change affect API or storage format -->
   
   ### Documentation
   
   <!-- Does this change introduce a new feature or require documentation updates -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(datafusion): support branch_manager and system table [paimon-rust]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #290:
URL: https://github.com/apache/paimon-rust/pull/290


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(datafusion): support branch_manager and system table [paimon-rust]

Posted by "xuzifu666 (via GitHub)" <gi...@apache.org>.
xuzifu666 commented on code in PR #290:
URL: https://github.com/apache/paimon-rust/pull/290#discussion_r3155079123


##########
crates/paimon/src/table/branch_manager.rs:
##########
@@ -0,0 +1,577 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Branch manager for reading and managing branch metadata using FileIO.
+//!
+//! Reference: [org.apache.paimon.utils.BranchManager](https://github.com/apache/paimon/blob/master/paimon-core/src/main/java/org/apache/paimon/utils/BranchManager.java)
+//! and [org.apache.paimon.utils.FileSystemBranchManager](https://github.com/apache/paimon/blob/master/paimon-core/src/main/java/org/apache/paimon/utils/FileSystemBranchManager.java)
+
+use crate::catalog::DEFAULT_MAIN_BRANCH;
+use crate::io::FileIO;
+use crate::table::{SchemaManager, SnapshotManager, TagManager};
+use opendal::raw::get_basename;
+
+const BRANCH_DIR: &str = "branch";
+const BRANCH_PREFIX: &str = "branch-";
+
+/// Manager for branch directories using unified FileIO.
+///
+/// Branches are stored as sub-directories at `{table_path}/branch/branch-{name}`.
+///
+/// Reference: [org.apache.paimon.utils.BranchManager](https://github.com/apache/paimon/blob/master/paimon-core/src/main/java/org/apache/paimon/utils/BranchManager.java)
+#[derive(Debug, Clone)]
+pub struct BranchManager {
+    file_io: FileIO,
+    table_path: String,
+}
+
+impl BranchManager {
+    pub fn new(file_io: FileIO, table_path: String) -> Self {
+        Self {
+            file_io,
+            table_path,
+        }
+    }
+
+    /// Path to the branch directory (e.g. `table_path/branch`).
+    pub fn branch_directory(&self) -> String {
+        format!("{}/{}", self.table_path, BRANCH_DIR)
+    }
+
+    /// Path to the branch sub-directory for the given name (e.g. `branch/branch-my_branch`).
+    pub fn branch_path(&self, branch_name: &str) -> String {
+        format!(
+            "{}/{}{}",
+            self.branch_directory(),
+            BRANCH_PREFIX,
+            branch_name
+        )
+    }
+
+    /// Validate branch name format.
+    ///
+    /// Rules:
+    /// - Cannot be "main"
+    /// - Cannot be blank or whitespace only
+    /// - Cannot be a pure numeric string
+    fn validate_branch_name(branch_name: &str) -> crate::Result<()> {
+        if branch_name == DEFAULT_MAIN_BRANCH {
+            return Err(crate::Error::DataInvalid {
+                message: format!(
+                    "Branch name '{}' is the default branch and cannot be used.",
+                    DEFAULT_MAIN_BRANCH
+                ),
+                source: None,
+            });
+        }
+        if branch_name.trim().is_empty() {
+            return Err(crate::Error::DataInvalid {
+                message: format!("Branch name '{}' is blank.", branch_name),
+                source: None,
+            });
+        }
+        if branch_name.chars().all(|c| c.is_ascii_digit()) {
+            return Err(crate::Error::DataInvalid {
+                message: format!(
+                    "Branch name cannot be pure numeric string but is '{}'.",
+                    branch_name
+                ),
+                source: None,
+            });
+        }
+        Ok(())
+    }
+
+    /// Validate branch name and ensure it does not already exist.
+    pub async fn validate_branch(&self, branch_name: &str) -> crate::Result<()> {
+        Self::validate_branch_name(branch_name)?;
+        if self.branch_exists(branch_name).await? {
+            return Err(crate::Error::DataInvalid {
+                message: format!("Branch name '{}' already exists.", branch_name),
+                source: None,
+            });
+        }
+        Ok(())
+    }
+
+    /// Check if a branch exists.
+    pub async fn branch_exists(&self, branch_name: &str) -> crate::Result<bool> {
+        let names = self.list_all().await?;

Review Comment:
   Yes, it has been changed to determine based on branch name.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(datafusion): support branch_manager and system table [paimon-rust]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #290:
URL: https://github.com/apache/paimon-rust/pull/290#discussion_r3154439289


##########
crates/paimon/src/table/branch_manager.rs:
##########
@@ -0,0 +1,577 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Branch manager for reading and managing branch metadata using FileIO.
+//!
+//! Reference: [org.apache.paimon.utils.BranchManager](https://github.com/apache/paimon/blob/master/paimon-core/src/main/java/org/apache/paimon/utils/BranchManager.java)
+//! and [org.apache.paimon.utils.FileSystemBranchManager](https://github.com/apache/paimon/blob/master/paimon-core/src/main/java/org/apache/paimon/utils/FileSystemBranchManager.java)
+
+use crate::catalog::DEFAULT_MAIN_BRANCH;
+use crate::io::FileIO;
+use crate::table::{SchemaManager, SnapshotManager, TagManager};
+use opendal::raw::get_basename;
+
+const BRANCH_DIR: &str = "branch";
+const BRANCH_PREFIX: &str = "branch-";
+
+/// Manager for branch directories using unified FileIO.
+///
+/// Branches are stored as sub-directories at `{table_path}/branch/branch-{name}`.
+///
+/// Reference: [org.apache.paimon.utils.BranchManager](https://github.com/apache/paimon/blob/master/paimon-core/src/main/java/org/apache/paimon/utils/BranchManager.java)
+#[derive(Debug, Clone)]
+pub struct BranchManager {
+    file_io: FileIO,
+    table_path: String,
+}
+
+impl BranchManager {
+    pub fn new(file_io: FileIO, table_path: String) -> Self {
+        Self {
+            file_io,
+            table_path,
+        }
+    }
+
+    /// Path to the branch directory (e.g. `table_path/branch`).
+    pub fn branch_directory(&self) -> String {
+        format!("{}/{}", self.table_path, BRANCH_DIR)
+    }
+
+    /// Path to the branch sub-directory for the given name (e.g. `branch/branch-my_branch`).
+    pub fn branch_path(&self, branch_name: &str) -> String {
+        format!(
+            "{}/{}{}",
+            self.branch_directory(),
+            BRANCH_PREFIX,
+            branch_name
+        )
+    }
+
+    /// Validate branch name format.
+    ///
+    /// Rules:
+    /// - Cannot be "main"
+    /// - Cannot be blank or whitespace only
+    /// - Cannot be a pure numeric string
+    fn validate_branch_name(branch_name: &str) -> crate::Result<()> {
+        if branch_name == DEFAULT_MAIN_BRANCH {
+            return Err(crate::Error::DataInvalid {
+                message: format!(
+                    "Branch name '{}' is the default branch and cannot be used.",
+                    DEFAULT_MAIN_BRANCH
+                ),
+                source: None,
+            });
+        }
+        if branch_name.trim().is_empty() {
+            return Err(crate::Error::DataInvalid {
+                message: format!("Branch name '{}' is blank.", branch_name),
+                source: None,
+            });
+        }
+        if branch_name.chars().all(|c| c.is_ascii_digit()) {
+            return Err(crate::Error::DataInvalid {
+                message: format!(
+                    "Branch name cannot be pure numeric string but is '{}'.",
+                    branch_name
+                ),
+                source: None,
+            });
+        }
+        Ok(())
+    }
+
+    /// Validate branch name and ensure it does not already exist.
+    pub async fn validate_branch(&self, branch_name: &str) -> crate::Result<()> {
+        Self::validate_branch_name(branch_name)?;
+        if self.branch_exists(branch_name).await? {
+            return Err(crate::Error::DataInvalid {
+                message: format!("Branch name '{}' already exists.", branch_name),
+                source: None,
+            });
+        }
+        Ok(())
+    }
+
+    /// Check if a branch exists.
+    pub async fn branch_exists(&self, branch_name: &str) -> crate::Result<bool> {
+        let names = self.list_all().await?;

Review Comment:
   Do not list all for `exists`, performance bad.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org