You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "JunRuiLee (via GitHub)" <gi...@apache.org> on 2026/06/26 11:06:10 UTC

[PR] [python] Expose DataFrame-style scan/split planning API [paimon-rust]

JunRuiLee opened a new pull request, #415:
URL: https://github.com/apache/paimon-rust/pull/415

   ### Purpose
   
   First PR of exposing PyPaimon's DataFrame read path to Rust. Refs #413.
   
   Exposes the Rust core's `ReadBuilder → new_scan() → plan() → DataSplit` flow
   through `bindings/python`:
   
   - `table.new_read_builder().with_projection([...]).with_limit(n).new_scan().plan()`
   - `Plan.splits()` returns `Split` objects that are **picklable** (opaque
     serde_json bytes), so they can cross process boundaries — groundwork for
     Ray distributed reads and a later `read(splits) → Arrow` step.
   
   Out of scope (later PRs): `with_filter` / Predicate conversion, `read(splits)`
   data reading, pypaimon-side wiring.
   
   ### Notes
   
   - `with_projection` is accepted and forwarded, but in PR1 it does **not** affect
     the planned splits — Rust core projection only applies to `new_read()` (data
     reading), not `new_scan()`/planning. It is staged here for the later
     `read(splits)` step. Tests assert only that planning succeeds.
   - `Split` is an opaque payload: the serde encoding is an implementation detail,
     not a stable schema; only same/compatible-version round-trip is guaranteed.
   
   ### Tests
   
   `bindings/python/tests/test_read.py` — 7 tests: scan planning
   (projection/limit/len) + pickle round-trip.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [python] Expose DataFrame-style scan/split planning API [paimon-rust]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #415:
URL: https://github.com/apache/paimon-rust/pull/415


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org