Re: [PR] [HUDI-7219] Add caching support for HFileBlocks [hudi]

["yihua (via GitHub)" <gi...@apache.org>]

yihua commented on code in PR #13724:
URL: https://github.com/apache/hudi/pull/13724#discussion_r2331884670


##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java:
##########
@@ -527,6 +527,33 @@ public final class HoodieMetadataConfig extends HoodieConfig {
       .withDocumentation("Max size in MB below which metadata file (HFile) will be downloaded "
           + "and cached entirely for the HFileReader.");
 
+  public static final ConfigProperty<Boolean> METADATA_HFILE_BLOCK_CACHE_ENABLED = ConfigProperty
+      .key(METADATA_PREFIX + ".hfile.block.cache.enabled")
+      .defaultValue(false)
+      .markAdvanced()
+      .sinceVersion("1.1.0")
+      .withDocumentation("Enable HFile block-level caching for metadata files. This caches frequently "
+          + "accessed HFile blocks in memory to reduce I/O operations during metadata queries. "
+          + "Improves performance for workloads with repeated metadata access patterns.");
+
+  public static final ConfigProperty<Integer> METADATA_HFILE_BLOCK_CACHE_SIZE = ConfigProperty
+      .key(METADATA_PREFIX + ".hfile.block.cache.size")
+      .defaultValue(100)
+      .markAdvanced()
+      .sinceVersion("1.1.0")
+      .withDocumentation("Maximum number of HFile blocks to cache in memory per metadata file reader. "
+          + "Higher values improve cache hit rates but consume more memory. "
+          + "Only effective when hfile.block.cache.enabled is true.");
+
+  public static final ConfigProperty<Integer> METADATA_HFILE_BLOCK_CACHE_TTL_MINUTES = ConfigProperty
+      .key(METADATA_PREFIX + ".hfile.block.cache.ttl.minutes")
+      .defaultValue(60)
+      .markAdvanced()
+      .sinceVersion("1.1.0")
+      .withDocumentation("Time-to-live (TTL) in minutes for cached HFile blocks. Blocks are evicted "

Review Comment:
   Does this account for the access pattern, i.e., if the same HFile block is access in an hour from t, t+1min, t+2min, ..., to t+59min, is the block going to be evicted from the block cache at t+60min, or it is evicted at t+119min?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org