You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@fluss.apache.org by Jim Hu <ya...@gmail.com> on 2025/08/08 12:01:43 UTC
[DISCUSS] FIP-11: Fluss Python binding on Fluss Rust
Hi Fluss community,
I’d like to kick off the public discussion on FIP-11¹, which proposes a
thin PyO3 wrapper around the existing Rust client so that Python users can
read/write Fluss without spinning up a JVM.
- Goal: Java-SDK parity (streaming & batch reads/writes, DDL, point
queries) but 100 % Pythonic.
- Implementation: PyO3 bridge compiled to a native .so/.pyd; zero-copy,
lock-free, no sockets, no JVM start-up cost.
- Repository: fluss-rust/bindings/python.
I’ve prototyped a minimal PoC at naivedogger/Fluss-Python-Client at
python-client
<https://github.com/naivedogger/Fluss-Python-Client/tree/python-client>
Looking forward to your thoughts.
Best regards,
Jim
[1]
https://cwiki.apache.org/confluence/display/FLUSS/FIP-11%3A+Fluss+Python+binding+on+Fluss+Rust
Re: [DISCUSS] FIP-11: Fluss Python binding on Fluss Rust
Posted by Jim Hu <ya...@gmail.com>.
Hi everyone,
I’ve updated the “Python Client API” section and also the accompanying example to match it. The connection config is now a Map, giving users a more flexible, dynamic setup.
Thanks for all the great feedback—if there are no further comments, I’ll open the vote shortly.
Best regards,
Jim
> 2025年8月12日 21:49，Jim Hu <ya...@gmail.com> 写道：
>
> Hi, Jark.
>
> Thanks for your detailed review. I'll address all of your concerns.
>
> 1. The FIP’s specification intentionally outlines the complete interface—including Arrow, Pandas, and DuckDB helpers—to ensure forward compatibility as the client evolves. The initial release, however, will be limited to bounded batch operations; it will expose only the scan method with an explicit end_timestamp and the corresponding to_arrow and to_pandas conversions.
>
> 2. AppendWriter.write_arrow is asynchronous. It returns a coroutine that completes only after the Arrow table is durably appended, similar to a Java CompletableFuture.
>
> 3. scan returns a lazy iterator that never materialises the entire log.
> to_arrow() collects the iterator into a single PyArrow Table—use it only when the data is small enough to fit in memory.
> to_arrow_batch() yields a RecordBatchReader that streams batches incrementally, keeping memory pressure low and matching the underlying Rust scanner’s streaming semantics.
>
> 4. The second parametre was intended to be a timeout in ms. However, after reviewing, we only have these configurations: bootstrap_server, request_max_size, writer_acks, writer_retries, and writer_batch_size. I have revised the Python demo to only contain the bootstrap_server.
>
> Best,
> Jim
>
> Jark Wu <imjark@gmail.com <ma...@gmail.com>> 于2025年8月12日周二 11:33写道：
>> Thanks Jim for proposing the python and rust client.
>>
>> The overall design looks good to me. +1 for Rust bindings (via PyO3 or
>> similar) over solutions like Py4J or a separate Python-native
>> implementation.
>>
>> This aligns with the current industry standard for high-performance Python
>> integrations, like Lance, Iceberg, and DeltaLake. Leveraging Rust bindings
>> will not only deliver better performance but also allow us to maximize code
>> reuse from the existing Rust client, reducing maintenance overhead and
>> ensuring consistency across language interfaces.
>>
>> Regarding the FIP, I have some following comments:
>>
>> 1. Could you clarify the intended scope of this FIP? Specifically, does it
>> include integration with PyArrow, Pandas, and DuckDB? The example shows
>> DuckDB integration, but it’s also listed under Future Work.
>>
>> 2. What's the semantics of AppendWriter#write_arrow? Is it a blocking API
>> or a non-blocking API? If it is a non-blocking API, can we have something
>> like Java CompletableFuture as the return type to allow users when the call
>> is successful?
>>
>> 3. What's the behavior of LogScanner#scan_earliest? Does it load all data
>> into client memory upfront and return a fully materialized ScanResult, or
>> does it support streaming semantics, fetching data incrementally to reduce
>> memory pressure?
>>
>> 4. What's the interface of `fluss.Config`? The config = fluss.Config("
>> 127.0.0.1:9123 <http://127.0.0.1:9123/>", 30) in the example confuses me about what the parameters
>> mean?
>>
>>
>> Best,
>> Jark
>>
>>
>>
>> On Fri, 8 Aug 2025 at 20:03, Jim Hu <yangyanghu365@gmail.com <ma...@gmail.com>> wrote:
>>
>> > Hi Fluss community,
>> > I’d like to kick off the public discussion on FIP-11¹, which proposes a
>> > thin PyO3 wrapper around the existing Rust client so that Python users can
>> > read/write Fluss without spinning up a JVM.
>> >
>> > - Goal: Java-SDK parity (streaming & batch reads/writes, DDL, point
>> > queries) but 100 % Pythonic.
>> > - Implementation: PyO3 bridge compiled to a native .so/.pyd; zero-copy,
>> > lock-free, no sockets, no JVM start-up cost.
>> > - Repository: fluss-rust/bindings/python.
>> >
>> > I’ve prototyped a minimal PoC at naivedogger/Fluss-Python-Client at
>> > python-client
>> > <https://github.com/naivedogger/Fluss-Python-Client/tree/python-client>
>> > Looking forward to your thoughts.
>> > Best regards,
>> > Jim
>> > [1]
>> >
>> > https://cwiki.apache.org/confluence/display/FLUSS/FIP-11%3A+Fluss+Python+binding+on+Fluss+Rust
>> >
Re: [DISCUSS] FIP-11: Fluss Python binding on Fluss Rust
Posted by Jim Hu <ya...@gmail.com>.
Hi everyone,
I’ve updated the “Python Client API” section and also the accompanying example to match it. The connection config is now a Map, giving users a more flexible, dynamic setup.
Thanks for all the great feedback—if there are no further comments, I’ll open the vote shortly.
Best regards,
Jim
> 2025年8月12日 21:49，Jim Hu <ya...@gmail.com> 写道：
>
> Hi, Jark.
>
> Thanks for your detailed review. I'll address all of your concerns.
>
> 1. The FIP’s specification intentionally outlines the complete interface—including Arrow, Pandas, and DuckDB helpers—to ensure forward compatibility as the client evolves. The initial release, however, will be limited to bounded batch operations; it will expose only the scan method with an explicit end_timestamp and the corresponding to_arrow and to_pandas conversions.
>
> 2. AppendWriter.write_arrow is asynchronous. It returns a coroutine that completes only after the Arrow table is durably appended, similar to a Java CompletableFuture.
>
> 3. scan returns a lazy iterator that never materialises the entire log.
> to_arrow() collects the iterator into a single PyArrow Table—use it only when the data is small enough to fit in memory.
> to_arrow_batch() yields a RecordBatchReader that streams batches incrementally, keeping memory pressure low and matching the underlying Rust scanner’s streaming semantics.
>
> 4. The second parametre was intended to be a timeout in ms. However, after reviewing, we only have these configurations: bootstrap_server, request_max_size, writer_acks, writer_retries, and writer_batch_size. I have revised the Python demo to only contain the bootstrap_server.
>
> Best,
> Jim
>
> Jark Wu <imjark@gmail.com <ma...@gmail.com>> 于2025年8月12日周二 11:33写道：
>> Thanks Jim for proposing the python and rust client.
>>
>> The overall design looks good to me. +1 for Rust bindings (via PyO3 or
>> similar) over solutions like Py4J or a separate Python-native
>> implementation.
>>
>> This aligns with the current industry standard for high-performance Python
>> integrations, like Lance, Iceberg, and DeltaLake. Leveraging Rust bindings
>> will not only deliver better performance but also allow us to maximize code
>> reuse from the existing Rust client, reducing maintenance overhead and
>> ensuring consistency across language interfaces.
>>
>> Regarding the FIP, I have some following comments:
>>
>> 1. Could you clarify the intended scope of this FIP? Specifically, does it
>> include integration with PyArrow, Pandas, and DuckDB? The example shows
>> DuckDB integration, but it’s also listed under Future Work.
>>
>> 2. What's the semantics of AppendWriter#write_arrow? Is it a blocking API
>> or a non-blocking API? If it is a non-blocking API, can we have something
>> like Java CompletableFuture as the return type to allow users when the call
>> is successful?
>>
>> 3. What's the behavior of LogScanner#scan_earliest? Does it load all data
>> into client memory upfront and return a fully materialized ScanResult, or
>> does it support streaming semantics, fetching data incrementally to reduce
>> memory pressure?
>>
>> 4. What's the interface of `fluss.Config`? The config = fluss.Config("
>> 127.0.0.1:9123 <http://127.0.0.1:9123/>", 30) in the example confuses me about what the parameters
>> mean?
>>
>>
>> Best,
>> Jark
>>
>>
>>
>> On Fri, 8 Aug 2025 at 20:03, Jim Hu <yangyanghu365@gmail.com <ma...@gmail.com>> wrote:
>>
>> > Hi Fluss community,
>> > I’d like to kick off the public discussion on FIP-11¹, which proposes a
>> > thin PyO3 wrapper around the existing Rust client so that Python users can
>> > read/write Fluss without spinning up a JVM.
>> >
>> > - Goal: Java-SDK parity (streaming & batch reads/writes, DDL, point
>> > queries) but 100 % Pythonic.
>> > - Implementation: PyO3 bridge compiled to a native .so/.pyd; zero-copy,
>> > lock-free, no sockets, no JVM start-up cost.
>> > - Repository: fluss-rust/bindings/python.
>> >
>> > I’ve prototyped a minimal PoC at naivedogger/Fluss-Python-Client at
>> > python-client
>> > <https://github.com/naivedogger/Fluss-Python-Client/tree/python-client>
>> > Looking forward to your thoughts.
>> > Best regards,
>> > Jim
>> > [1]
>> >
>> > https://cwiki.apache.org/confluence/display/FLUSS/FIP-11%3A+Fluss+Python+binding+on+Fluss+Rust
>> >
Re: [DISCUSS] FIP-11: Fluss Python binding on Fluss Rust
Posted by Jim Hu <ya...@gmail.com>.
Hi, Jark.
Thanks for your detailed review. I'll address all of your concerns.
1. The FIP’s specification intentionally outlines the complete
interface—including Arrow, Pandas, and DuckDB helpers—to ensure forward
compatibility as the client evolves. The initial release, however, will be
limited to bounded batch operations; it will expose only the scan method
with an explicit end_timestamp and the corresponding to_arrow and to_pandas
conversions.
2. AppendWriter.write_arrow is asynchronous. It returns a coroutine that
completes only after the Arrow table is durably appended, similar to a Java
CompletableFuture.
3. scan returns a lazy iterator that never materialises the entire log.
to_arrow() collects the iterator into a single PyArrow Table—use it only
when the data is small enough to fit in memory.
to_arrow_batch() yields a RecordBatchReader that streams batches
incrementally, keeping memory pressure low and matching the underlying Rust
scanner’s streaming semantics.
4. The second parametre was intended to be a timeout in ms. However, after
reviewing, we only have these configurations: bootstrap_server,
request_max_size, writer_acks, writer_retries, and writer_batch_size. I
have revised the Python demo to only contain the bootstrap_server.
Best,
Jim
Jark Wu <im...@gmail.com> 于2025年8月12日周二 11:33写道：
> Thanks Jim for proposing the python and rust client.
>
> The overall design looks good to me. +1 for Rust bindings (via PyO3 or
> similar) over solutions like Py4J or a separate Python-native
> implementation.
>
> This aligns with the current industry standard for high-performance Python
> integrations, like Lance, Iceberg, and DeltaLake. Leveraging Rust bindings
> will not only deliver better performance but also allow us to maximize code
> reuse from the existing Rust client, reducing maintenance overhead and
> ensuring consistency across language interfaces.
>
> Regarding the FIP, I have some following comments:
>
> 1. Could you clarify the intended scope of this FIP? Specifically, does it
> include integration with PyArrow, Pandas, and DuckDB? The example shows
> DuckDB integration, but it’s also listed under Future Work.
>
> 2. What's the semantics of AppendWriter#write_arrow? Is it a blocking API
> or a non-blocking API? If it is a non-blocking API, can we have something
> like Java CompletableFuture as the return type to allow users when the call
> is successful?
>
> 3. What's the behavior of LogScanner#scan_earliest? Does it load all data
> into client memory upfront and return a fully materialized ScanResult, or
> does it support streaming semantics, fetching data incrementally to reduce
> memory pressure?
>
> 4. What's the interface of `fluss.Config`? The config = fluss.Config("
> 127.0.0.1:9123", 30) in the example confuses me about what the parameters
> mean?
>
>
> Best,
> Jark
>
>
>
> On Fri, 8 Aug 2025 at 20:03, Jim Hu <ya...@gmail.com> wrote:
>
> > Hi Fluss community,
> > I’d like to kick off the public discussion on FIP-11¹, which proposes a
> > thin PyO3 wrapper around the existing Rust client so that Python users
> can
> > read/write Fluss without spinning up a JVM.
> >
> > - Goal: Java-SDK parity (streaming & batch reads/writes, DDL, point
> > queries) but 100 % Pythonic.
> > - Implementation: PyO3 bridge compiled to a native .so/.pyd;
> zero-copy,
> > lock-free, no sockets, no JVM start-up cost.
> > - Repository: fluss-rust/bindings/python.
> >
> > I’ve prototyped a minimal PoC at naivedogger/Fluss-Python-Client at
> > python-client
> > <https://github.com/naivedogger/Fluss-Python-Client/tree/python-client>
> > Looking forward to your thoughts.
> > Best regards,
> > Jim
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-11%3A+Fluss+Python+binding+on+Fluss+Rust
> >
>
Re: [DISCUSS] FIP-11: Fluss Python binding on Fluss Rust
Posted by Jim Hu <ya...@gmail.com>.
Hi, Yuxia.
Thanks for your comments.
#1. Good catch! I've consolidated the four legacy scan methods into a
single scan(**kw) entry point. By passing different keyword arguments users
can obtain all four original behaviors.
#2. Thanks for the suggestion.
yuxia <lu...@alumni.sjtu.edu.cn> 于2025年8月12日周二 15:12写道：
> Thanks Jim for driving this work. Provide Python client on Rust binding is
> nice idea. Some comments on the Python api design:
>
> 1: In LogScanner, there're two methods with same name, def
> scan_earliest(self, end_timestamp: long) -> ScanResult:, def
> scan_earliest(self) -> ScanResult:
> But IIUC, python doesn't support method override, so two methods with same
> name but different parameters won't work.
>
> 2: I'd suggest to rename `pollBatch` in BatchScanner to `poll_batch` to
> align the naming style.
>
>
>
> Best regards,
> Yuxia
>
> ----- 原始邮件 -----
> 发件人: "Jark Wu" <im...@gmail.com>
> 收件人: "dev" <de...@fluss.apache.org>
> 发送时间: 星期二, 2025年 8 月 12日 上午 11:32:53
> 主题: [SPAM]Re: [DISCUSS] FIP-11: Fluss Python binding on Fluss Rust
>
> Thanks Jim for proposing the python and rust client.
>
> The overall design looks good to me. +1 for Rust bindings (via PyO3 or
> similar) over solutions like Py4J or a separate Python-native
> implementation.
>
> This aligns with the current industry standard for high-performance Python
> integrations, like Lance, Iceberg, and DeltaLake. Leveraging Rust bindings
> will not only deliver better performance but also allow us to maximize code
> reuse from the existing Rust client, reducing maintenance overhead and
> ensuring consistency across language interfaces.
>
> Regarding the FIP, I have some following comments:
>
> 1. Could you clarify the intended scope of this FIP? Specifically, does it
> include integration with PyArrow, Pandas, and DuckDB? The example shows
> DuckDB integration, but it’s also listed under Future Work.
>
> 2. What's the semantics of AppendWriter#write_arrow? Is it a blocking API
> or a non-blocking API? If it is a non-blocking API, can we have something
> like Java CompletableFuture as the return type to allow users when the call
> is successful?
>
> 3. What's the behavior of LogScanner#scan_earliest? Does it load all data
> into client memory upfront and return a fully materialized ScanResult, or
> does it support streaming semantics, fetching data incrementally to reduce
> memory pressure?
>
> 4. What's the interface of `fluss.Config`? The config = fluss.Config("
> 127.0.0.1:9123", 30) in the example confuses me about what the parameters
> mean?
>
>
> Best,
> Jark
>
>
>
> On Fri, 8 Aug 2025 at 20:03, Jim Hu <ya...@gmail.com> wrote:
>
> > Hi Fluss community,
> > I’d like to kick off the public discussion on FIP-11¹, which proposes a
> > thin PyO3 wrapper around the existing Rust client so that Python users
> can
> > read/write Fluss without spinning up a JVM.
> >
> > - Goal: Java-SDK parity (streaming & batch reads/writes, DDL, point
> > queries) but 100 % Pythonic.
> > - Implementation: PyO3 bridge compiled to a native .so/.pyd;
> zero-copy,
> > lock-free, no sockets, no JVM start-up cost.
> > - Repository: fluss-rust/bindings/python.
> >
> > I’ve prototyped a minimal PoC at naivedogger/Fluss-Python-Client at
> > python-client
> > <https://github.com/naivedogger/Fluss-Python-Client/tree/python-client>
> > Looking forward to your thoughts.
> > Best regards,
> > Jim
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-11%3A+Fluss+Python+binding+on+Fluss+Rust
> >
>
Re: [DISCUSS] FIP-11: Fluss Python binding on Fluss Rust
Posted by yuxia <lu...@alumni.sjtu.edu.cn>.
Thanks Jim for driving this work. Provide Python client on Rust binding is nice idea. Some comments on the Python api design:
1: In LogScanner, there're two methods with same name, def scan_earliest(self, end_timestamp: long) -> ScanResult:, def scan_earliest(self) -> ScanResult:
But IIUC, python doesn't support method override, so two methods with same name but different parameters won't work.
2: I'd suggest to rename `pollBatch` in BatchScanner to `poll_batch` to align the naming style.
Best regards,
Yuxia
----- 原始邮件 -----
发件人: "Jark Wu" <im...@gmail.com>
收件人: "dev" <de...@fluss.apache.org>
发送时间: 星期二, 2025年 8 月 12日 上午 11:32:53
主题: [SPAM]Re: [DISCUSS] FIP-11: Fluss Python binding on Fluss Rust
Thanks Jim for proposing the python and rust client.
The overall design looks good to me. +1 for Rust bindings (via PyO3 or
similar) over solutions like Py4J or a separate Python-native
implementation.
This aligns with the current industry standard for high-performance Python
integrations, like Lance, Iceberg, and DeltaLake. Leveraging Rust bindings
will not only deliver better performance but also allow us to maximize code
reuse from the existing Rust client, reducing maintenance overhead and
ensuring consistency across language interfaces.
Regarding the FIP, I have some following comments:
1. Could you clarify the intended scope of this FIP? Specifically, does it
include integration with PyArrow, Pandas, and DuckDB? The example shows
DuckDB integration, but it’s also listed under Future Work.
2. What's the semantics of AppendWriter#write_arrow? Is it a blocking API
or a non-blocking API? If it is a non-blocking API, can we have something
like Java CompletableFuture as the return type to allow users when the call
is successful?
3. What's the behavior of LogScanner#scan_earliest? Does it load all data
into client memory upfront and return a fully materialized ScanResult, or
does it support streaming semantics, fetching data incrementally to reduce
memory pressure?
4. What's the interface of `fluss.Config`? The config = fluss.Config("
127.0.0.1:9123", 30) in the example confuses me about what the parameters
mean?
Best,
Jark
On Fri, 8 Aug 2025 at 20:03, Jim Hu <ya...@gmail.com> wrote:
> Hi Fluss community,
> I’d like to kick off the public discussion on FIP-11¹, which proposes a
> thin PyO3 wrapper around the existing Rust client so that Python users can
> read/write Fluss without spinning up a JVM.
>
> - Goal: Java-SDK parity (streaming & batch reads/writes, DDL, point
> queries) but 100 % Pythonic.
> - Implementation: PyO3 bridge compiled to a native .so/.pyd; zero-copy,
> lock-free, no sockets, no JVM start-up cost.
> - Repository: fluss-rust/bindings/python.
>
> I’ve prototyped a minimal PoC at naivedogger/Fluss-Python-Client at
> python-client
> <https://github.com/naivedogger/Fluss-Python-Client/tree/python-client>
> Looking forward to your thoughts.
> Best regards,
> Jim
> [1]
>
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-11%3A+Fluss+Python+binding+on+Fluss+Rust
>
Re: [DISCUSS] FIP-11: Fluss Python binding on Fluss Rust
Posted by Jark Wu <im...@gmail.com>.
Thanks Jim for proposing the python and rust client.
The overall design looks good to me. +1 for Rust bindings (via PyO3 or
similar) over solutions like Py4J or a separate Python-native
implementation.
This aligns with the current industry standard for high-performance Python
integrations, like Lance, Iceberg, and DeltaLake. Leveraging Rust bindings
will not only deliver better performance but also allow us to maximize code
reuse from the existing Rust client, reducing maintenance overhead and
ensuring consistency across language interfaces.
Regarding the FIP, I have some following comments:
1. Could you clarify the intended scope of this FIP? Specifically, does it
include integration with PyArrow, Pandas, and DuckDB? The example shows
DuckDB integration, but it’s also listed under Future Work.
2. What's the semantics of AppendWriter#write_arrow? Is it a blocking API
or a non-blocking API? If it is a non-blocking API, can we have something
like Java CompletableFuture as the return type to allow users when the call
is successful?
3. What's the behavior of LogScanner#scan_earliest? Does it load all data
into client memory upfront and return a fully materialized ScanResult, or
does it support streaming semantics, fetching data incrementally to reduce
memory pressure?
4. What's the interface of `fluss.Config`? The config = fluss.Config("
127.0.0.1:9123", 30) in the example confuses me about what the parameters
mean?
Best,
Jark
On Fri, 8 Aug 2025 at 20:03, Jim Hu <ya...@gmail.com> wrote:
> Hi Fluss community,
> I’d like to kick off the public discussion on FIP-11¹, which proposes a
> thin PyO3 wrapper around the existing Rust client so that Python users can
> read/write Fluss without spinning up a JVM.
>
> - Goal: Java-SDK parity (streaming & batch reads/writes, DDL, point
> queries) but 100 % Pythonic.
> - Implementation: PyO3 bridge compiled to a native .so/.pyd; zero-copy,
> lock-free, no sockets, no JVM start-up cost.
> - Repository: fluss-rust/bindings/python.
>
> I’ve prototyped a minimal PoC at naivedogger/Fluss-Python-Client at
> python-client
> <https://github.com/naivedogger/Fluss-Python-Client/tree/python-client>
> Looking forward to your thoughts.
> Best regards,
> Jim
> [1]
>
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-11%3A+Fluss+Python+binding+on+Fluss+Rust
>
Re: [DISCUSS] FIP-11: Fluss Python binding on Fluss Rust
Posted by Jim Hu <ya...@gmail.com>.
Hi Michael,
Thanks for your insightful suggestions, here are my thoughts.
1. Minimum Python Version: Good point. Let’s support Python >= 3.9, which aligns with the Flink Python client, and also eliminates any EOL concerns for 3.7.
2. Tools(uv, ruff, and mypy): Nice suggestion! As for now, in the first version of Python client, I think there will be basically no hand-written Python to lint, except for a few simple examples. However, I think these tools are great, and we can introduce them later on.
Best,
Jim
> 2025年8月17日 22:03，Michael Koepf <mi...@apache.org> 写道：
>
> Hi Jim,
>
> Thank you for bringing Fluss to the Python ecosystem!
>
> Since this is the first Python project within Fluss, a few (late) questions from my side around project setup and ecosystem which have not been addressed yet.
>
> - What is the minimum Python version that will be supported? In the PoC it seems to be 3.7 [1]. However, Python 3.7 is already end-of-life [2]. Maybe we should opt for 3.9, similar to the Flink Python client [3], to keep potential maintenance overhead low?
>
> - What do you think about using uv [4] for package and project management? It is compatible with the maturin build backend and offers integration with tools for linting and static type checking.
>
> - I am not sure how much non-generated Python code there will be in the first version (maybe it's even just generated code from PyO3?). But what tools do you plan to use for linting and static type checking? Personally, I would prefer ruff [5] and mypy [6], but people might have different opinions.
>
> Thank you.
>
> --
> Best,
> MKO
>
> [1] https://github.com/naivedogger/Fluss-Python-Client/blob/4a93bcd3c57db1c3ea7022e10106df73c3aff1fd/python-bindings/pyproject.toml#L10
> [2] https://devguide.python.org/versions/
> [3] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/python/datastream_tutorial/
> [4] https://docs.astral.sh/uv/
> [5] https://docs.astral.sh/ruff/
> [6] https://mypy-lang.org/
>
> On 2025/08/08 12:01:43 Jim Hu wrote:
>> Hi Fluss community,
>> I’d like to kick off the public discussion on FIP-11¹, which proposes a
>> thin PyO3 wrapper around the existing Rust client so that Python users can
>> read/write Fluss without spinning up a JVM.
>>
>> - Goal: Java-SDK parity (streaming & batch reads/writes, DDL, point
>> queries) but 100 % Pythonic.
>> - Implementation: PyO3 bridge compiled to a native .so/.pyd; zero-copy,
>> lock-free, no sockets, no JVM start-up cost.
>> - Repository: fluss-rust/bindings/python.
>>
>> I’ve prototyped a minimal PoC at naivedogger/Fluss-Python-Client at
>> python-client
>> <https://github.com/naivedogger/Fluss-Python-Client/tree/python-client>
>> Looking forward to your thoughts.
>> Best regards,
>> Jim
>> [1]
>> https://cwiki.apache.org/confluence/display/FLUSS/FIP-11%3A+Fluss+Python+binding+on+Fluss+Rust
Re: [DISCUSS] FIP-11: Fluss Python binding on Fluss Rust
Posted by Michael Koepf <mi...@apache.org>.
Hi Jim,
Thank you for bringing Fluss to the Python ecosystem!
Since this is the first Python project within Fluss, a few (late) questions from my side around project setup and ecosystem which have not been addressed yet.
- What is the minimum Python version that will be supported? In the PoC it seems to be 3.7 [1]. However, Python 3.7 is already end-of-life [2]. Maybe we should opt for 3.9, similar to the Flink Python client [3], to keep potential maintenance overhead low?
- What do you think about using uv [4] for package and project management? It is compatible with the maturin build backend and offers integration with tools for linting and static type checking.
- I am not sure how much non-generated Python code there will be in the first version (maybe it's even just generated code from PyO3?). But what tools do you plan to use for linting and static type checking? Personally, I would prefer ruff [5] and mypy [6], but people might have different opinions.
Thank you.
--
Best,
MKO
[1] https://github.com/naivedogger/Fluss-Python-Client/blob/4a93bcd3c57db1c3ea7022e10106df73c3aff1fd/python-bindings/pyproject.toml#L10
[2] https://devguide.python.org/versions/
[3] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/python/datastream_tutorial/
[4] https://docs.astral.sh/uv/
[5] https://docs.astral.sh/ruff/
[6] https://mypy-lang.org/
On 2025/08/08 12:01:43 Jim Hu wrote:
> Hi Fluss community,
> I’d like to kick off the public discussion on FIP-11¹, which proposes a
> thin PyO3 wrapper around the existing Rust client so that Python users can
> read/write Fluss without spinning up a JVM.
>
> - Goal: Java-SDK parity (streaming & batch reads/writes, DDL, point
> queries) but 100 % Pythonic.
> - Implementation: PyO3 bridge compiled to a native .so/.pyd; zero-copy,
> lock-free, no sockets, no JVM start-up cost.
> - Repository: fluss-rust/bindings/python.
>
> I’ve prototyped a minimal PoC at naivedogger/Fluss-Python-Client at
> python-client
> <https://github.com/naivedogger/Fluss-Python-Client/tree/python-client>
> Looking forward to your thoughts.
> Best regards,
> Jim
> [1]
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-11%3A+Fluss+Python+binding+on+Fluss+Rust
>