You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by "rishabhdaim (via GitHub)" <gi...@apache.org> on 2026/05/15 18:28:42 UTC
[PR] OAK-12216 : clear segment cache after successful compaction [jackrabbit-oak]
rishabhdaim opened a new pull request, #2904:
URL: https://github.com/apache/jackrabbit-oak/pull/2904
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] OAK-12216 : clear segment cache after successful compaction [jackrabbit-oak]
Posted by "ChlineSaurus (via GitHub)" <gi...@apache.org>.
ChlineSaurus commented on PR #2904:
URL: https://github.com/apache/jackrabbit-oak/pull/2904#issuecomment-4475353827
What AI flagged (I'm not familiar with this part of the code, so I would need some time to read up on it, to give a proper review. Since this was flagged as the mosts risky PR, I'm still adding it):
After a successful compaction (Oak's GC), this PR throws away the entire L2 cache so the new generation of segments can fill it from scratch.
Why it's problematic:
- It's redundant. The cleanup step that runs after every compaction already clears the cache. This PR adds a second clear earlier in the pipeline. Net result: two cold-cache windows where there used to
be one. That's strictly worse for hit rate.
- It directly contradicts existing code. Another compaction strategy has an explicit comment: "we don't clear the segment cache. There might be transient segments in there." The PR ignores this.
- Wrong order. It clears L2 before the listener that flushes old-generation segments from the writer cache. In the gap, concurrent commits can refill L2 with the very old-gen segments we just tried to
evict.
- Partial-success path clears too. Even when compaction only partly succeeded, it still wipes the cache — including the segments that were just committed and about to be read.
- Races with concurrent putSegment. Clearing while a writer is mid-insert can leave L1 references pointing at "unloaded" state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] OAK-12216 : clear segment cache after successful compaction [jackrabbit-oak]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #2904:
URL: https://github.com/apache/jackrabbit-oak/pull/2904#issuecomment-4476591511
## [](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [91.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] OAK-12216 : clear segment cache after successful compaction [jackrabbit-oak]
Posted by "rishabhdaim (via GitHub)" <gi...@apache.org>.
rishabhdaim commented on PR #2904:
URL: https://github.com/apache/jackrabbit-oak/pull/2904#issuecomment-4474561391
Build is failing due to resource leakage in _**DocumentNodeStoreIT**_ test. To be fixed : https://github.com/apache/jackrabbit-oak/pull/2905
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] OAK-12216 : clear segment cache after successful compaction [jackrabbit-oak]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #2904:
URL: https://github.com/apache/jackrabbit-oak/pull/2904#issuecomment-4485245480
## [](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] OAK-12216 : clear segment cache after successful compaction [jackrabbit-oak]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #2904:
URL: https://github.com/apache/jackrabbit-oak/pull/2904#issuecomment-4477269911
## [](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [100.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=org.apache.jackrabbit%3Ajackrabbit-oak&pullRequest=2904)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] OAK-12216 : clear segment cache after successful compaction [jackrabbit-oak]
Posted by "rishabhdaim (via GitHub)" <gi...@apache.org>.
rishabhdaim commented on PR #2904:
URL: https://github.com/apache/jackrabbit-oak/pull/2904#issuecomment-4477479303
@jsedding Yes, I agree with you on this.
Also, based on the benchmarks, the improvement is only for first few thousand entries only.
After sometime, caffeine, starts performing better than Guava even without changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] OAK-12216 : clear segment cache after successful compaction [jackrabbit-oak]
Posted by "rishabhdaim (via GitHub)" <gi...@apache.org>.
rishabhdaim commented on PR #2904:
URL: https://github.com/apache/jackrabbit-oak/pull/2904#issuecomment-4485614312
As discussed with @jsedding we won't be fixing this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] OAK-12216 : clear segment cache after successful compaction [jackrabbit-oak]
Posted by "rishabhdaim (via GitHub)" <gi...@apache.org>.
rishabhdaim closed pull request #2904: OAK-12216 : clear segment cache after successful compaction
URL: https://github.com/apache/jackrabbit-oak/pull/2904
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] OAK-12216 : clear segment cache after successful compaction [jackrabbit-oak]
Posted by "rishabhdaim (via GitHub)" <gi...@apache.org>.
rishabhdaim commented on PR #2904:
URL: https://github.com/apache/jackrabbit-oak/pull/2904#issuecomment-4475600231
thanks @ChlineSaurus for the review. I have asked re-review for the AI suggested review and below are the findings:
Verdict on each AI claim
> Claim 1: "Cleanup already clears the cache — this adds a redundant second cold-cache window"
Mostly correct, but overstated. _DefaultCleanupStrategy.cleanup()_ (line 41) does unconditionally call _context.getSegmentCache().clear()_. The PR adds a second clear in _notifyCompactionSucceeded →
AbstractCompactionStrategy.java:82_. So there are now two clears per GC cycle.
However "strictly worse" overstates it. The first clear's intent is legitimate: evict old-gen segments before new-gen reads start filling the cache, so new-gen content doesn't compete with stale entries. The
problem is that cleanup immediately follows and clears everything again anyway — so anything loaded between the two clears is thrown away. This is a real but mild design issue, not catastrophic.
> Claim 2: "Contradicts CleanupFirstCompactionStrategy's 'transient segments' comment"
**Misleading**. That comment is in _CleanupFirstCompactionStrategy.java:60_ and refers to the pre-compaction cleanup phase — when the new generation hasn't been committed yet and transient (in-flight) segments exist.
The PR's clear happens after _setHead()_ committed the new generation. These are different phases with different semantics. The "transient segments" concern simply doesn't apply post-commit.
> Claim 3: "Wrong order — clears L2 before L1 old-gen eviction"
Technically true but practically negligible. The _cacheManager.evictOldGeneration()_ call happens inside _compactionSucceeded()_, which fires after the new code's cache clear. But after _setHead()_ commits the new
generation, concurrent writers already use the new generation. The window between the two operations where old-gen segments could sneak back into L2 is real but vanishingly small.
> Claim 4: "Partial-success path clears too"
**Valid**. _notifyCompactionSucceeded()_ is called from both _compactionSucceeded()_ (line 288) and _compactionPartiallySucceeded()_ (line 292). For partial success, segments from the new partial generation were just
committed and are about to be read — clearing them from L2 hurts without clear benefit. This is analogous to why _CleanupFirstCompactionStrategy_ explicitly avoids clearing: those partially-compacted segments are
live data. This is the strongest point in the review.
> Claim 5: "Race between clear() and concurrent putSegment()"
Overstated. The _putSegment()_ implementation calls _id.loaded(segment)_ before _cache.put(id, segment)_. If a concurrent _clear()_ fires between those two calls, the L1 memoization _(SegmentId.segment)_ gets cleared by
_onRemove() → id.unloaded()_. This results in an L1 miss on the next access, not data corruption — the segment is still loaded from L2 or disk correctly. Moreover, this same race already existed before this PR from
the _cleanup's clear()_. Not a new issue.
P.S. : The one claim worth acting on is _#4: notifyCompactionSucceeded()_ should probably skip the cache clear when _!compacted.isComplete() (partial success)_.
The double-clear concern (#1) is also worth discussing with reviewers — either justify why the early clear adds value despite cleanup doing the same, or remove it and rely solely on cleanup's existing clear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: oak-dev-unsubscribe@jackrabbit.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org