[I] [lake] Skip snapshot recovery when offsets belong to a previous table ID [fluss]

["luoyuxia (via GitHub)" <gi...@apache.org>]

luoyuxia opened a new issue, #4821:
URL: https://github.com/apache/fluss/issues/4821

   ### Search before asking
   
   - [ ] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
   
   Related: #3483 and its merged PR #3494 address stale coordinator commits around table deletion. This issue tracks the remaining recovery path: a live, recreated table can recover a lake snapshot whose offsets belong to the previous table ID.
   
   ### Fluss version
   
   main (development), inspected at 5d4bea8c6.
   
   ### Please describe the bug 🐞
   
   When a Fluss table is dropped and recreated while reusing the same Paimon table, the old Paimon snapshots retain the previous Fluss table's offsets-file property. If the new table starts tiering without registering an initial lake snapshot, `getMissingLakeSnapshot(null)` can return one of these old snapshots.
   
   `TieringCommitter.checkMissingLakeSnapshotAndRecover` currently registers the returned snapshot and offsets path under the current table ID without checking the `table_id` stored in the offsets file. The existing check that the committable's table ID matches the current Fluss table does not protect this path: the committable already belongs to the new table.
   
   Triggering sequence:
   
   1. Tier data from Fluss table `db.t` (table ID A) into Paimon, producing snapshot S with an offsets file for A.
   2. Drop the Fluss table, retain the Paimon table, and recreate `db.t` with table ID B.
   3. Reuse the Paimon table and start tiering without the initial-snapshot registration performed by the promotion flow. Keep the old offsets file accessible to isolate this recovery bug from asynchronous file cleanup.
   4. The recovery check selects S and attempts to register A's offsets path under B.
   
   Expected: offsets belonging to A must not be registered under B; the current table should continue its normal commit.
   
   Actual: old snapshot metadata can be attached to B. For a non-partitioned table, reading the snapshot can then expose the old bucket offsets as B's progress, causing new data to be skipped or tiering to stall.
   
   ### Solution
   
   Before registering a missing lake snapshot, deserialize its offsets file with `TableBucketOffsets` and compare its table ID with the current Fluss table ID. If they differ, log a warning containing the table path, both IDs, snapshot ID, and offsets path; skip recovery and continue the current commit.
   
   Preserve recovery for matching table IDs, including the existing abort/retry behavior after a partially completed commit. This change does not add special handling for missing, unreadable, or malformed offsets files.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [lake] Skip snapshot recovery when offsets belong to a previous table ID [fluss]

["beryllw (via GitHub)" <gi...@apache.org>]

beryllw closed issue #4821: [lake] Skip snapshot recovery when offsets belong to a previous table ID
URL: https://github.com/apache/fluss/issues/4821


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org