You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fluss.apache.org by "luoyuxia (via GitHub)" <gi...@apache.org> on 2026/08/27 01:17:20 UTC
[I] [FIP-28] Complete end-to-end historical partition writes [fluss]
luoyuxia opened a new issue, #4119:
URL: https://github.com/apache/fluss/issues/4119
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Description
Parent issue: #3631
Follow-up to #3633 and #4001. PR #4001 added the server-side primary-key write path for the shared historical partition, but intentionally deferred client-side routing, log-table writes, historical recovery and tiering, and cleanup of local historical KV overlays.
Complete the end-to-end write path for historical partitions in datalake-enabled tables. The implementation should cover:
- Resolve potentially expired target partitions before a write batch is queued, route confirmed historical writes through the internal historical partition, and preserve the original physical partition selected for each batch.
- Fail pending writes when refreshed metadata confirms that the original target partition no longer exists, while retaining the existing retry behavior for ordinary writes.
- Carry original partition names through both `PUT_KV` and `PRODUCE_LOG`, with API-version gates and correct response correlation.
- Tier historical KV and log records back to their original Paimon partitions, including recovery and lookup-after-write behavior.
- Clean fully tiered historical KV overlays only after the lake and local log end offsets match. Guard queued cleanup with the leader epoch and scheduled log end offset, and support dynamic idle-time configuration updates.
- Add unit and integration coverage for client routing, KV and log writes, Paimon tiering, recovery, missing targets, and overlay cleanup.
This issue is separate from #3633 because it tracks the client and lake integration required to make the server-side write path usable end to end, plus the lifecycle work explicitly deferred by #4001.
### Willingness to contribute
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [FIP-28] Complete end-to-end historical partition writes [fluss]
Posted by "wuchong (via GitHub)" <gi...@apache.org>.
wuchong closed issue #4119: [FIP-28] Complete end-to-end historical partition writes
URL: https://github.com/apache/fluss/issues/4119
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org