You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "JingsongLi (via GitHub)" <gi...@apache.org> on 2026/09/06 19:07:23 UTC
[PR] [core] Unify block cache ownership and avoid redundant SST reads [paimon]
JingsongLi opened a new pull request, #9663:
URL: https://github.com/apache/paimon/pull/9663
### Purpose
Centralize cached file-page ownership in `CacheManager` so readers share a single authoritative cache. Remove reader-local page maps, detach evicted pages from the file index, and invalidate by page identity so an older invalidation cannot delete a concurrently loaded replacement.
Reduce redundant work in SST lookups:
- Read a block and its trailer together, cache the decoded range, and charge the complete retained allocation while avoiding the uncompressed heap copy.
- Use non-loading cache probes on hits, preserve atomic loading on misses, and record accesses through Caffeine's normal eviction policy.
- Probe resident Bloom filters without I/O and use cached data directly when the filter is absent. After admission rejection, choose cold reads using their cost and allow bounded retries so the same reader can recover when cache pressure eases.
- Keep Bloom probes stateless with respect to page ownership and ensure reader cleanup attempts all relevant invalidations.
### Tests
Passed 389 related tests on JDK 8, with zero failures, errors, or skipped tests. The standard Maven lifecycle also passed Checkstyle, Spotless, and Enforcer checks.
```sh
mvn -o -pl paimon-common -DwildcardSuites=none \
'-Dtest=BlockCacheTest,CacheManager*Test,CaffeineCacheTest,SstFileReaderCacheTest,FileBasedBloomFilterTest,BloomFilterTest,BitSetTest,SortLookupStore*Test,LocalKvDb*Test,BTreeIndexReader*Test,BTreeThreadSafetyTest,LazyFilteredBTreeIndexReaderTest' test
```
Regression coverage includes rejected-page cleanup, concurrent loading and invalidation, shared-reader reloads, heap/off-heap allocation accounting, CRC failure and retry, Bloom read costs, and recovery after rejected admission.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Unify block cache ownership and avoid redundant SST reads [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #9663:
URL: https://github.com/apache/paimon/pull/9663
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org