You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "JunRuiLee (via GitHub)" <gi...@apache.org> on 2026/06/26 11:06:10 UTC
[PR] [python] Expose DataFrame-style scan/split planning API [paimon-rust]
JunRuiLee opened a new pull request, #415:
URL: https://github.com/apache/paimon-rust/pull/415
### Purpose
First PR of exposing PyPaimon's DataFrame read path to Rust. Refs #413.
Exposes the Rust core's `ReadBuilder → new_scan() → plan() → DataSplit` flow
through `bindings/python`:
- `table.new_read_builder().with_projection([...]).with_limit(n).new_scan().plan()`
- `Plan.splits()` returns `Split` objects that are **picklable** (opaque
serde_json bytes), so they can cross process boundaries — groundwork for
Ray distributed reads and a later `read(splits) → Arrow` step.
Out of scope (later PRs): `with_filter` / Predicate conversion, `read(splits)`
data reading, pypaimon-side wiring.
### Notes
- `with_projection` is accepted and forwarded, but in PR1 it does **not** affect
the planned splits — Rust core projection only applies to `new_read()` (data
reading), not `new_scan()`/planning. It is staged here for the later
`read(splits)` step. Tests assert only that planning succeeds.
- `Split` is an opaque payload: the serde encoding is an implementation detail,
not a stable schema; only same/compatible-version round-trip is guaranteed.
### Tests
`bindings/python/tests/test_read.py` — 7 tests: scan planning
(projection/limit/len) + pickle round-trip.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [python] Expose DataFrame-style scan/split planning API [paimon-rust]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #415:
URL: https://github.com/apache/paimon-rust/pull/415
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org