You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "tub (via GitHub)" <gi...@apache.org> on 2026/03/04 14:30:01 UTC
[PR] [python] Add StreamReadBuilder and AsyncStreamingTableScan [paimon]
tub opened a new pull request, #7343:
URL: https://github.com/apache/paimon/pull/7343
## Summary
PR 2 of 3 for pure-Python streaming reads. This PR adds the core streaming API:
- **StreamReadBuilder** for configuring streaming reads with predicates, projections, consumer IDs, sharding, and poll intervals
- **AsyncStreamingTableScan** with async generator and sync wrapper for continuous snapshot polling
- **Table interface additions**: `new_stream_read_builder()` on Table, FileStoreTable, FormatTable (stub), IcebergTable (stub)
- **setup.py**: add `oss`/`lance` extras
- **Documentation**: streaming reads section and supported features list
11 files changed, +2287 / -2 lines (incremental)
## PR Stack
1. [Streaming infrastructure](https://github.com/apache/paimon/pull/7342) (scanners, consumers, caching, sharding)
2. **👉 this PR** — Core streaming (StreamReadBuilder, AsyncStreamingTableScan, table integration)
3. CLI (`paimon tail` command)
**Incremental diff** (just this PR's changes): https://github.com/apache/paimon/compare/python-streaming-1-infra...tub:paimon:python-streaming-2-core
**Merge workflow:** Merge PR 1, rebase PR 2 onto updated master (PR 1 commits drop out), merge PR 2, repeat for PR 3.
## Test plan
- [x] `python -m pytest pypaimon/tests` — 594 passed (9 pre-existing lance failures)
- [x] `python -c "from pypaimon import CatalogFactory"` — no import errors
- [x] Unit tests for StreamReadBuilder validation, sharding, and AsyncStreamingTableScan filtering
- [x] Integration tests for stream_read_builder shard passthrough and FileScanner filtering
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [python] Add StreamReadBuilder and AsyncStreamingTableScan [paimon]
Posted by "tub (via GitHub)" <gi...@apache.org>.
tub closed pull request #7343: [python] Add StreamReadBuilder and AsyncStreamingTableScan
URL: https://github.com/apache/paimon/pull/7343
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [python] Add StreamReadBuilder and AsyncStreamingTableScan [paimon]
Posted by "tub (via GitHub)" <gi...@apache.org>.
tub commented on PR #7343:
URL: https://github.com/apache/paimon/pull/7343#issuecomment-4006323928
## ⚠️ Superseded
This PR has been replaced by:
- #7350 — StreamReadBuilder + AsyncStreamingTableScan
Which now stacks on the split infrastructure PRs (#7347, #7348, #7349).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [python] Add StreamReadBuilder and AsyncStreamingTableScan [paimon]
Posted by "tub (via GitHub)" <gi...@apache.org>.
tub commented on PR #7343:
URL: https://github.com/apache/paimon/pull/7343#issuecomment-4005786313
Superseded by python-streaming-2-core-v2 which stacks on split infra PRs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org