You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@paimon.apache.org by yuxia <lu...@alumni.sjtu.edu.cn> on 2026/01/10 03:22:39 UTC
DuckDB Extension for Apache Paimon - Community Interest and Collaboration
Dear Paimon Community,
I hope this message finds you well. I'm writing to share a project I've been working on and to gauge community interest in it.
I've developed a DuckDB extension for Apache Paimon that enables direct querying of Paimon tables using DuckDB. This extension allows users to quickly explore and analyze Paimon data locally without the need to set up and configure larger distributed engines like Trino or StarRocks. This makes it particularly useful for:
- Quick data exploration and ad-hoc analysis
- Prototyping and development workflows
- Local testing and validation
- Lowering the barrier to entry for Paimon adoption
**Project Details:**
- Repository: https://github.com/luoyuxia/duckdb-extension-paimon
- Current Status: Experimental (0.0.1-beta release available for macOS, tested with DuckDB 1.4.2)
- Supported Table Types: Log tables and primary key tables with Deletion Vector enabled
- Technology Stack: Built on top of paimon-rust with a Rust FFI layer and C++ integration
**Technical Approach:**
The extension is built using paimon-rust as its foundation. I've been maintaining a fork of paimon-rust (https://github.com/luoyuxia/paimon-rust, poc branch) that adds read capabilities, which serves as the core for this DuckDB extension. This approach leverages Rust's performance and safety characteristics while providing a clean integration path with DuckDB through FFI.
**Future Plans:**
If the community finds this project valuable and interesting, I would be very happy to:
1. Continue refining and improving the DuckDB extension for better integration with DuckDB's features and ecosystem
2. Contribute the read capabilities back to the paimon-rust project, which could unlock more integrations between Paimon and the broader Rust ecosystem
I'm eager to hear your thoughts and would welcome any feedback, suggestions, or collaboration opportunities.
Thank you for your time and consideration. I look forward to hearing from the community.
Best regards,
Yuxia
Re: DuckDB Extension for Apache Paimon - Community Interest and Collaboration
Posted by yuxia <lu...@alumni.sjtu.edu.cn>.
Hi Jingsong, thanks for the warm welcome!
I'm fully committed to the continuous development of paimon-rust. Integrating it with DuckDB is a fantastic idea. I'm actually planning to use paimon-rust as the foundation to unlock DuckDB's ability to "union read" Fluss and Paimon data together. This could be a game-changer for hybrid real-time/batch analytics.
I will keep you updated on my progress!
Best regards,
Yuxia
----- 原始邮件 -----
发件人: "Jingsong Li" <ji...@gmail.com>
收件人: "user" <us...@paimon.apache.org>
抄送: "dev" <de...@paimon.apache.org>
发送时间: 星期一, 2026年 1 月 12日 上午 10:44:20
主题: Re: DuckDB Extension for Apache Paimon - Community Interest and Collaboration
Hi Yuxia, Thank you for your excellent work!
I'm thrilled that paimon-rust has appeared in a real-life scenario!
Welcome to contribute.
If possible, you can contribute `duckdb-extension-paimon` too. Maybe
in paimon-rust, maybe another repo.
Best,
Jingsong
On Sat, Jan 10, 2026 at 11:23 AM yuxia <lu...@alumni.sjtu.edu.cn> wrote:
>
> Dear Paimon Community,
>
> I hope this message finds you well. I'm writing to share a project I've been working on and to gauge community interest in it.
>
> I've developed a DuckDB extension for Apache Paimon that enables direct querying of Paimon tables using DuckDB. This extension allows users to quickly explore and analyze Paimon data locally without the need to set up and configure larger distributed engines like Trino or StarRocks. This makes it particularly useful for:
> - Quick data exploration and ad-hoc analysis
> - Prototyping and development workflows
> - Local testing and validation
> - Lowering the barrier to entry for Paimon adoption
>
> **Project Details:**
> - Repository: https://github.com/luoyuxia/duckdb-extension-paimon
> - Current Status: Experimental (0.0.1-beta release available for macOS, tested with DuckDB 1.4.2)
> - Supported Table Types: Log tables and primary key tables with Deletion Vector enabled
> - Technology Stack: Built on top of paimon-rust with a Rust FFI layer and C++ integration
>
> **Technical Approach:**
> The extension is built using paimon-rust as its foundation. I've been maintaining a fork of paimon-rust (https://github.com/luoyuxia/paimon-rust, poc branch) that adds read capabilities, which serves as the core for this DuckDB extension. This approach leverages Rust's performance and safety characteristics while providing a clean integration path with DuckDB through FFI.
>
> **Future Plans:**
> If the community finds this project valuable and interesting, I would be very happy to:
> 1. Continue refining and improving the DuckDB extension for better integration with DuckDB's features and ecosystem
> 2. Contribute the read capabilities back to the paimon-rust project, which could unlock more integrations between Paimon and the broader Rust ecosystem
>
> I'm eager to hear your thoughts and would welcome any feedback, suggestions, or collaboration opportunities.
>
> Thank you for your time and consideration. I look forward to hearing from the community.
>
> Best regards,
> Yuxia
Re: DuckDB Extension for Apache Paimon - Community Interest and Collaboration
Posted by yuxia <lu...@alumni.sjtu.edu.cn>.
Hi Jingsong, thanks for the warm welcome!
I'm fully committed to the continuous development of paimon-rust. Integrating it with DuckDB is a fantastic idea. I'm actually planning to use paimon-rust as the foundation to unlock DuckDB's ability to "union read" Fluss and Paimon data together. This could be a game-changer for hybrid real-time/batch analytics.
I will keep you updated on my progress!
Best regards,
Yuxia
----- 原始邮件 -----
发件人: "Jingsong Li" <ji...@gmail.com>
收件人: "user" <us...@paimon.apache.org>
抄送: "dev" <de...@paimon.apache.org>
发送时间: 星期一, 2026年 1 月 12日 上午 10:44:20
主题: Re: DuckDB Extension for Apache Paimon - Community Interest and Collaboration
Hi Yuxia, Thank you for your excellent work!
I'm thrilled that paimon-rust has appeared in a real-life scenario!
Welcome to contribute.
If possible, you can contribute `duckdb-extension-paimon` too. Maybe
in paimon-rust, maybe another repo.
Best,
Jingsong
On Sat, Jan 10, 2026 at 11:23 AM yuxia <lu...@alumni.sjtu.edu.cn> wrote:
>
> Dear Paimon Community,
>
> I hope this message finds you well. I'm writing to share a project I've been working on and to gauge community interest in it.
>
> I've developed a DuckDB extension for Apache Paimon that enables direct querying of Paimon tables using DuckDB. This extension allows users to quickly explore and analyze Paimon data locally without the need to set up and configure larger distributed engines like Trino or StarRocks. This makes it particularly useful for:
> - Quick data exploration and ad-hoc analysis
> - Prototyping and development workflows
> - Local testing and validation
> - Lowering the barrier to entry for Paimon adoption
>
> **Project Details:**
> - Repository: https://github.com/luoyuxia/duckdb-extension-paimon
> - Current Status: Experimental (0.0.1-beta release available for macOS, tested with DuckDB 1.4.2)
> - Supported Table Types: Log tables and primary key tables with Deletion Vector enabled
> - Technology Stack: Built on top of paimon-rust with a Rust FFI layer and C++ integration
>
> **Technical Approach:**
> The extension is built using paimon-rust as its foundation. I've been maintaining a fork of paimon-rust (https://github.com/luoyuxia/paimon-rust, poc branch) that adds read capabilities, which serves as the core for this DuckDB extension. This approach leverages Rust's performance and safety characteristics while providing a clean integration path with DuckDB through FFI.
>
> **Future Plans:**
> If the community finds this project valuable and interesting, I would be very happy to:
> 1. Continue refining and improving the DuckDB extension for better integration with DuckDB's features and ecosystem
> 2. Contribute the read capabilities back to the paimon-rust project, which could unlock more integrations between Paimon and the broader Rust ecosystem
>
> I'm eager to hear your thoughts and would welcome any feedback, suggestions, or collaboration opportunities.
>
> Thank you for your time and consideration. I look forward to hearing from the community.
>
> Best regards,
> Yuxia
Re: DuckDB Extension for Apache Paimon - Community Interest and Collaboration
Posted by Jingsong Li <ji...@gmail.com>.
Hi Yuxia, Thank you for your excellent work!
I'm thrilled that paimon-rust has appeared in a real-life scenario!
Welcome to contribute.
If possible, you can contribute `duckdb-extension-paimon` too. Maybe
in paimon-rust, maybe another repo.
Best,
Jingsong
On Sat, Jan 10, 2026 at 11:23 AM yuxia <lu...@alumni.sjtu.edu.cn> wrote:
>
> Dear Paimon Community,
>
> I hope this message finds you well. I'm writing to share a project I've been working on and to gauge community interest in it.
>
> I've developed a DuckDB extension for Apache Paimon that enables direct querying of Paimon tables using DuckDB. This extension allows users to quickly explore and analyze Paimon data locally without the need to set up and configure larger distributed engines like Trino or StarRocks. This makes it particularly useful for:
> - Quick data exploration and ad-hoc analysis
> - Prototyping and development workflows
> - Local testing and validation
> - Lowering the barrier to entry for Paimon adoption
>
> **Project Details:**
> - Repository: https://github.com/luoyuxia/duckdb-extension-paimon
> - Current Status: Experimental (0.0.1-beta release available for macOS, tested with DuckDB 1.4.2)
> - Supported Table Types: Log tables and primary key tables with Deletion Vector enabled
> - Technology Stack: Built on top of paimon-rust with a Rust FFI layer and C++ integration
>
> **Technical Approach:**
> The extension is built using paimon-rust as its foundation. I've been maintaining a fork of paimon-rust (https://github.com/luoyuxia/paimon-rust, poc branch) that adds read capabilities, which serves as the core for this DuckDB extension. This approach leverages Rust's performance and safety characteristics while providing a clean integration path with DuckDB through FFI.
>
> **Future Plans:**
> If the community finds this project valuable and interesting, I would be very happy to:
> 1. Continue refining and improving the DuckDB extension for better integration with DuckDB's features and ecosystem
> 2. Contribute the read capabilities back to the paimon-rust project, which could unlock more integrations between Paimon and the broader Rust ecosystem
>
> I'm eager to hear your thoughts and would welcome any feedback, suggestions, or collaboration opportunities.
>
> Thank you for your time and consideration. I look forward to hearing from the community.
>
> Best regards,
> Yuxia
Re: DuckDB Extension for Apache Paimon - Community Interest and Collaboration
Posted by Jingsong Li <ji...@gmail.com>.
Hi Yuxia, Thank you for your excellent work!
I'm thrilled that paimon-rust has appeared in a real-life scenario!
Welcome to contribute.
If possible, you can contribute `duckdb-extension-paimon` too. Maybe
in paimon-rust, maybe another repo.
Best,
Jingsong
On Sat, Jan 10, 2026 at 11:23 AM yuxia <lu...@alumni.sjtu.edu.cn> wrote:
>
> Dear Paimon Community,
>
> I hope this message finds you well. I'm writing to share a project I've been working on and to gauge community interest in it.
>
> I've developed a DuckDB extension for Apache Paimon that enables direct querying of Paimon tables using DuckDB. This extension allows users to quickly explore and analyze Paimon data locally without the need to set up and configure larger distributed engines like Trino or StarRocks. This makes it particularly useful for:
> - Quick data exploration and ad-hoc analysis
> - Prototyping and development workflows
> - Local testing and validation
> - Lowering the barrier to entry for Paimon adoption
>
> **Project Details:**
> - Repository: https://github.com/luoyuxia/duckdb-extension-paimon
> - Current Status: Experimental (0.0.1-beta release available for macOS, tested with DuckDB 1.4.2)
> - Supported Table Types: Log tables and primary key tables with Deletion Vector enabled
> - Technology Stack: Built on top of paimon-rust with a Rust FFI layer and C++ integration
>
> **Technical Approach:**
> The extension is built using paimon-rust as its foundation. I've been maintaining a fork of paimon-rust (https://github.com/luoyuxia/paimon-rust, poc branch) that adds read capabilities, which serves as the core for this DuckDB extension. This approach leverages Rust's performance and safety characteristics while providing a clean integration path with DuckDB through FFI.
>
> **Future Plans:**
> If the community finds this project valuable and interesting, I would be very happy to:
> 1. Continue refining and improving the DuckDB extension for better integration with DuckDB's features and ecosystem
> 2. Contribute the read capabilities back to the paimon-rust project, which could unlock more integrations between Paimon and the broader Rust ecosystem
>
> I'm eager to hear your thoughts and would welcome any feedback, suggestions, or collaboration opportunities.
>
> Thank you for your time and consideration. I look forward to hearing from the community.
>
> Best regards,
> Yuxia