[jira] [Created] (RATIS-2680) SimpleStateMachineStorage.cleanupOldSnapshots should consider that consider applications have created legit snapshots without MD5 files

["Rui Wang (Jira)" <ji...@apache.org>]

Rui Wang created RATIS-2680:
-------------------------------

             Summary: SimpleStateMachineStorage.cleanupOldSnapshots should consider that consider applications have created legit snapshots without MD5 files
                 Key: RATIS-2680
                 URL: https://issues.apache.org/jira/browse/RATIS-2680
             Project: Ratis
          Issue Type: Improvement
            Reporter: Rui Wang
            Assignee: Rui Wang


Follow-up to #1320 (RATIS-244), which changed `SimpleStateMachineStorage.cleanupOldSnapshots` to count only snapshots with MD5 files toward the retention limit. Meanwhile RATIS-244 maintained the backward compatibility for `SimpleStateMachineStorage.findLatestSnapshot`.

Overall speaking, there could be applications never considered creating MD5 files for taking snapshots, but because of old implementation of the Ratis, those applications didn't know that their snapshots should be considered incomplete or corrupted. Ratis should consider such applications and maintain backward compatibility. Otherwise once those applications upgrade, they will surprisingly find the behavior changes of the snapshot.

That change broke backward compatibility during upgrades where old snapshots (created before MD5 support, without .md5 files) coexist with new snapshots (with MD5 files) or there is no new snapshots (with MD5 files). In those cases, cleanup could either skip deletion entirely or retain the wrong set of snapshots, because old snapshots were excluded from the retention count even though they were valid before the upgrade.

This PR restores the original retention behavior as a fallback: when fewer than numSnapshotsRetained MD5 snapshots are found, retain the newest numSnapshotsRetained snapshots regardless of whether they have MD5 files. This covers:

1. All old snapshots without MD5 — retention policy applies as it did before RATIS-244
2. Mixed old and new snapshots after upgrade — old snapshots count toward retention when there are not enough new MD5 snapshots to fill the quota
3. When enough MD5 snapshots exist, the RATIS-244 behavior is unchanged: retention is based on MD5 snapshots, and snapshots without MD5 that fall beyond that cutoff are still cleaned up.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)