You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "keith-turner (via GitHub)" <gi...@apache.org> on 2023/07/03 19:19:22 UTC

[GitHub] [accumulo] keith-turner commented on a diff in pull request #3563: Reliably refresh tablets after updating tablet files for a compaction.

keith-turner commented on code in PR #3563:
URL: https://github.com/apache/accumulo/pull/3563#discussion_r1251214798


##########
server/manager/src/main/java/org/apache/accumulo/manager/compaction/coordinator/CompactionCoordinator.java:
##########
@@ -172,9 +198,62 @@ protected void startRunningCleaner(ScheduledThreadPoolExecutor schedExecutor) {
     ThreadPools.watchNonCriticalScheduledTask(future);
   }
 
+  private void processRefreshes(Ample.DataLevel dataLevel) {
+
+    // process batches of refresh entries to avoid reading all into memory at once
+    Iterators.partition(ctx.getAmple().refreshes(dataLevel).list().iterator(), 10000)
+        .forEachRemaining(refreshEntries -> {
+          LOG.info("Processing {} tablet refreshes for {}", refreshEntries.size(), dataLevel);
+
+          var extents =
+              refreshEntries.stream().map(RefreshEntry::getExtent).collect(Collectors.toList());
+          var tabletsMeta = new HashMap<KeyExtent,TabletMetadata>();
+          ctx.getAmple().readTablets().forTablets(extents, Optional.empty())
+              .fetch(PREV_ROW, LOCATION, SCANS).build().stream()

Review Comment:
   Fixed in dedaa88



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org