You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fluss.apache.org by "yangshangqing95 (via GitHub)" <gi...@apache.org> on 2026/10/16 22:13:03 UTC

[I] [RPC] Make log_start_offset authoritative in FetchLog v1 [fluss]

yangshangqing95 opened a new issue, #4677:
URL: https://github.com/apache/fluss/issues/4677

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
   
   
   ### Motivation
   
   ### Description
   `PbFetchLogRespForBucket` already defines `log_start_offset` (field 6), but its current value is not authoritative. The server historically populates it with `0`, so clients cannot reliably use it to determine the actual log start offset.
   
   PR #4545 introduces an optional `resolved_earliest_offset` (field 11) to support bounded scans starting from `EARLIEST_OFFSET`.
   
   This field provides the resolution information required by bounded scanning without changing the existing FetchLog v0 behavior.
   
   However, maintaining two overlapping offset fields is not ideal in the long term. We should eventually establish `log_start_offset` as the authoritative source of log-start information and consolidate the client implementation around it.
   
   ### Solution
   
   ### Proposed direction
   
   Introduce FetchLog v1 with well defined `log_start_offset` semantics and the necessary per call protocol version support.
   
   The migration should cover the following areas.
   
   **1. Define authoritative `log_start_offset` semantics**
   
   Specify what `log_start_offset` represents
   
   **2. Introduce FetchLog v1**
   
   Add the required protocol versioning and per-call version handling so that v1 responses can populate `log_start_offset` with its defined authoritative value.
   
   FetchLog v0 must retain its existing compatibility behavior.
   
   **3. Migrate client offset handling**
   
   For FetchLog v1, clients should use the authoritative `log_start_offset` instead of relying on `resolved_earliest_offset`, once the v1 semantics are sufficient to support bounded EARLIEST resolution.
   
   The implementation should preserve compatibility with older servers that only support FetchLog v0.
   
   **4. Deprecate and eventually reserve field 11**
   
   Once clients and servers have migrated to the authoritative v1 field:
   
   - Stop relying on `resolved_earliest_offset` for FetchLog v1.
   - Deprecate the temporary field when compatibility permits.
   - Eventually remove its declaration and reserve field number 11 (and its name) to prevent accidental reuse.
   
   Field 11 must remain available for existing protocol compatibility until it is safe to remove.
   
   ### Anything else?
   
   ### Compatibility considerations
   
   The migration should explicitly cover:
   
   | Client | Server | Expected behavior |
   |---|---|---|
   | Existing v0 client | v0 server | Preserve existing behavior |
   | Existing v0 client | v1-capable server | Continue using compatible v0 responses |
   | v1-capable client | v0 server | Use the supported v0 compatibility path |
   | v1-capable client | v1-capable server | Use authoritative `log_start_offset` |
   
   The protocol version must provide a reliable way to distinguish an authoritative offset of `0` from the historical placeholder value sent by v0 servers.
   
   ### Related
   
   Follow-up to #4545, which introduced `resolved_earliest_offset` as a backward-compatible solution for bounded EARLIEST scanning.
   
   The broader FetchLog protocol migration is intentionally outside the scope of that PR.
   
   ### Willingness to contribute
   
   - [x] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org