You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2022/03/25 21:21:49 UTC

[GitHub] [accumulo] keith-turner commented on a change in pull request #2592: Configure block cache for Recovery

keith-turner commented on a change in pull request #2592:
URL: https://github.com/apache/accumulo/pull/2592#discussion_r835616582



##########
File path: server/tserver/src/main/java/org/apache/accumulo/tserver/log/RecoveryLogsIterator.java
##########
@@ -79,7 +86,8 @@ public RecoveryLogsIterator(ServerContext context, List<Path> recoveryLogDirs, L
 
       for (Path log : logFiles) {
         var scanner = RFile.newScanner().from(log.toString()).withFileSystem(fs)
-            .withTableProperties(context.getConfiguration()).build();
+            .withTableProperties(context.getConfiguration()).withDataCache(dataCacheSize)

Review comment:
       This cache is per recovery log iterator.  This does not help with the case where tablet server has 100 tablets that need to recover data from the same walogs.  To help with this case would need to create dedicated block caches for recovery at the tsever level and pass them into this iterator.  Probably would want their own config so they can be size differently than the block caches for scans.  Also probably would not want to use the scan block cache for this as it will always be disjoint files and we probably do not want recovery to impact the caches used for scans by causing evictions.

##########
File path: server/tserver/src/main/java/org/apache/accumulo/tserver/log/RecoveryLogsIterator.java
##########
@@ -61,6 +65,9 @@
    */
   public RecoveryLogsIterator(ServerContext context, List<Path> recoveryLogDirs, LogFileKey start,
       LogFileKey end, boolean checkFirstKey) throws IOException {
+    var conf = context.getConfiguration();
+    var indexCacheSize = getMemoryAsBytes(conf.get(Property.TSERV_INDEXCACHE_SIZE));

Review comment:
       May want dedicated props for the recovery cache size.  For example I may want a 10G cache for scans and 256M cache for recovery.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org