You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-issues@jackrabbit.apache.org by "Rishabh Daim (Jira)" <ji...@apache.org> on 2026/09/29 16:22:00 UTC
[jira] [Commented] (OAK-12408) Standalone cleanup() never journals gc.log leading to SNFE on cold standby / backup
[ https://issues.apache.org/jira/browse/OAK-12408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18120815#comment-18120815 ]
Rishabh Daim commented on OAK-12408:
------------------------------------
lgtm, thanks [~shivamaggarwal] for the contribution.
> Standalone cleanup() never journals gc.log leading to SNFE on cold standby / backup
> -----------------------------------------------------------------------------------
>
> Key: OAK-12408
> URL: https://issues.apache.org/jira/browse/OAK-12408
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
> Affects Versions: 2.6.0
> Reporter: Shivam
> Assignee: Rishabh Daim
> Priority: Critical
> Fix For: 2.8.0
>
>
> {{AbstractGarbageCollectionStrategy.cleanup(Context)}} can run without a preceding local compaction (e.g. {{StandbyClientSync}} after sync, {{{}FileStoreBackupImpl{}}}/{{{}FileStoreBackupRestoreMBean{}}}). In that case it builds a placeholder via {{{}CompactionResult.skipped(...){}}}, whose {{requiresGCJournalEntry()}} is hard-coded to {{{}false{}}}:
> {code:java}
> static CompactionResult skipped(...) {
> return new CompactionResult(currentGeneration, gcCount) {
> // requiresGCJournalEntry() not overridden -> always false
> };
> }
> {code}
> So {{GCJournal.persist(...)}} is never called for this path, even though the reclaimer *is* actively removing segments from older generations. On a cold standby, {{currentGeneration}} legitimately advances on every synced generation, but {{gc.log}} stays frozen and keeps pointing at a stale, already-superseded root.
> h3. Impact
> * Standby's own {{gc.log}} never gains new entries after boot, even though sync + cleanup keep reclaiming segments from newer generations.
> * After promoting the standby to primary, later compaction/cleanup cycles trust this stale {{gc.log}} entry, and reads of already-reclaimed segments throw {{{}SegmentNotFoundException{}}}.
> h3. Steps to reproduce
> # Set up primary + cold standby.
> # Run several rounds of content + online cleanup on primary so multiple generations/gc.log entries exist.
> # Standby's {{segmentstore/gc.log}} shows no new entries despite "cleanup completed... space reclaimed" logs and on-disk generation clearly advancing.
> # Promote standby to primary, keep running cleanup — eventually {{SegmentNotFoundException}} on segments already reclaimed while {{gc.log}} was frozen.
> h3. Suggested fix
> {code:java}
> @Override
> boolean requiresGCJournalEntry() {
> return currentGeneration.isCompacted();
> }
> {code}
> * {{GCJournal.persist}} already no-ops when the generation is unchanged, so this can't cause duplicate/bloated entries.
> * Fresh, never-compacted repos are unaffected ({{{}isCompacted(){}}} is false at generation 0).
> * {{{}succeeded(){}}}/{{{}aborted(){}}} paths untouched.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)