You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "VladRodionov (via GitHub)" <gi...@apache.org> on 2026/06/09 04:22:07 UTC
[PR] HBASE-30023 Refactor write path, prefetch, and compaction cache popul… [hbase]
VladRodionov opened a new pull request, #8325:
URL: https://github.com/apache/hbase/pull/8325
### Summary
This PR continues the pluggable block cache refactoring by migrating cache population paths to use CacheAccessService instead of calling BlockCache directly where the code is part of HFile cache access behavior.
The change keeps the runtime behavior legacy-compatible. Existing BlockCache implementations are still used through BlockCacheBackedCacheAccessService. This PR does not refactor BlockCacheFactory, does not introduce CacheEngine-backed runtime wiring, and does not attempt to migrate all remaining BlockCache references.
The main goal is to make CacheAccessService the HBase-facing facade for cache lookup, insertion, invalidation, admission checks, and related cache operations while preserving existing behavior.
### Changes
* Migrates additional cache population code paths from direct BlockCache usage to CacheAccessService.
* Preserves simple cacheBlock(...) overloads on CacheAccessService so legacy-style call sites do not need to construct CacheWriteContext manually.
* Keeps CacheWriteContext available for future richer policy-aware cache insertion paths.
* Adds/uses compatibility forwarding for cache admission hooks such as shouldCacheBlock(...) and shouldCacheFile(...) where existing paths depend on them.
* Adds/uses CacheAccessService.ifEnabled(...) for call sites that need the old Optional#ifPresent(...) style behavior when cache is disabled.
* Preserves disabled-cache behavior through NoOpCacheAccessService.
* Updates affected tests.
### Compatibility notes
This PR intentionally does not force every cache insertion call site to specify a precise CacheWriteSource.
Existing HBase cache population code does not always expose whether a block insertion comes from flush, compaction, bulk load, or another writer path. Guessing that source could introduce incorrect policy behavior later. Therefore, this PR keeps compatibility-oriented overloads and uses richer context only where the information is already available and meaningful.
### Out of scope
* No BlockCacheFactory refactoring.
* No CacheEngine adapter migration.
* No migration of LruBlockCache, LruAdaptiveBlockCache, TinyLfuBlockCache, or BucketCache to CacheEngine.
* No removal of BlockCache from CacheConfig.
* No removal of getBlockCache().
* No migration of BlockCache.iterator().
* No diagnostic JSP/admin page migration.
* No broad migration of implementation-specific cache tests.
* No topology-backed runtime wiring.
### Testing
Ran the affected HFile cache-on-write test successfully:
bash mvn -pl hbase-server -Dtest=TestHFile#testWriterCacheOnWriteSkipDoesNotLeak test
Also verified build/package after resolving a stale local class output issue unrelated to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-30023 Refactor write path, prefetch, and compaction cache popul… [hbase]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #8325:
URL: https://github.com/apache/hbase/pull/8325#discussion_r3406089791
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/cache/CacheAccessService.java:
##########
@@ -444,4 +448,72 @@ default void notifyFileCachingCompleted(Path fileName, int totalBlockCount, int
long size) {
// noop
}
+
+ /**
+ * Executes the supplied action when this cache service is enabled.
+ * <p>
+ * This helper is intended to preserve the old {@code getBlockCache().ifPresent(...)} style for
+ * call sites that should do nothing when block cache is disabled. It keeps callers independent of
+ * concrete implementations such as {@code NoOpCacheAccessService} while still allowing disabled
+ * cache wiring to behave like an absent cache.
+ * </p>
+ * <p>
+ * Implementations normally do not need to override this method. The default implementation checks
+ * {@link #isCacheEnabled()} and invokes the supplied action only when cache access is enabled.
+ * </p>
+ * @param action action to execute with this cache service when enabled
+ */
+ default void ifEnabled(Consumer<CacheAccessService> action) {
+ Objects.requireNonNull(action, "action must not be null");
+ if (isCacheEnabled()) {
+ action.accept(this);
+ }
+ }
+
+ /**
+ * Returns whether blocks from the given HFile should be cached. TODO: this method is a temporary
+ * adapter for file-level admission decisions. It will be removed and replaced by a more general
+ * admission API in the future.
+ * <p>
+ * This is a file-level admission hook used by cache population paths. Implementations may use
+ * file metadata, configuration, data tiering state, or implementation-specific bookkeeping to
+ * decide whether the file should be admitted into cache.
+ * </p>
+ * <p>
+ * The returned {@link Optional} is empty when the cache service does not support file-level
+ * admission decisions. In that case, callers should preserve existing default behavior, typically
+ * treating the result as "no opinion" rather than as a rejection.
+ * </p>
+ * @param hFileInfo HFile metadata used for the admission decision
+ * @param conf configuration
+ * @return empty if unsupported; otherwise whether the file should be cached
+ */
+ default Optional<Boolean> shouldCacheFile(HFileInfo hFileInfo, Configuration conf) {
+ return Optional.empty();
+ }
+
+ /**
+ * Returns whether the block represented by the given key and timestamp should be cached. TODO:
+ * this method is a temporary adapter for file-level admission decisions. It will be removed and
+ * replaced by a more general admission API in the future.
+ * <p>
+ * <p>
+ * This is a block-level admission hook used by cache population paths. Implementations may use
Review Comment:
Javadoc for shouldCacheBlock has a copy/paste error (mentions "file-level" admission for a block-level hook) and contains a duplicate `<p>` tag, which makes the generated Javadoc confusing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-30023 Refactor write path, prefetch, and compaction cache popul… [hbase]
Posted by "taklwu (via GitHub)" <gi...@apache.org>.
taklwu merged PR #8325:
URL: https://github.com/apache/hbase/pull/8325
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-30023 Refactor write path, prefetch, and compaction cache popul… [hbase]
Posted by "taklwu (via GitHub)" <gi...@apache.org>.
taklwu commented on code in PR #8325:
URL: https://github.com/apache/hbase/pull/8325#discussion_r3406063310
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/cache/CacheWriteSource.java:
##########
@@ -35,6 +35,16 @@ public enum CacheWriteSource {
*/
READ_MISS,
+ /**
+ * Block was inserted from the HFile write/cache-on-write path.
+ * <p>
+ * This source covers write-side population where the caller does not expose whether the HFile is
+ * being produced by flush, compaction, bulk load, or another writer. More specific write-side
+ * sources can be added later when that context is available at the call site.
+ * </p>
+ */
+ WRITE_PATH,
+
Review Comment:
nit: I got this is a new type, but in this PR, we didn't use it and IMO we should add new type when it's being used?
##########
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestLazyDataBlockDecompression.java:
##########
@@ -132,6 +132,10 @@ private static void cacheBlocks(Configuration conf, CacheConfig cacheConfig, Fil
reader.close();
}
+ /*
+ * TODO: migrate this test to use new HBASE-30018 APIs and remove the need to cast to
+ * LruBlockCache
+ */
Review Comment:
nit: if you have plan to update it , we can have this in the upcoming JIRA without this reminder.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org