[GitHub] [hbase] saintstack commented on a change in pull request #1022: HBASE-23680 RegionProcedureStore missing cleaning of hfile archive

[GitBox <gi...@apache.org>]

saintstack commented on a change in pull request #1022: HBASE-23680 RegionProcedureStore missing cleaning of hfile archive
URL: https://github.com/apache/hbase/pull/1022#discussion_r365531590
 
 

 ##########
 File path: hbase-server/src/main/java/org/apache/hadoop/hbase/procedure2/store/region/RegionFlusherAndCompactor.java
 ##########
 @@ -136,9 +146,30 @@ static void setupConf(Configuration conf) {
       flushPerChanges, flushIntervalMs);
   }
 
+  private void deleteCompactedHFiles() throws IOException {
+    HStore store = Iterables.getOnlyElement(region.getStores());
+    store.closeAndArchiveCompactedFiles();
+    // for now we just deleted these HFiles, without moving them to the global archive directory.
+    // This is because that, our HFiles are on the WAL file system, but the global HFile archive
+    // directory is on the root(HFile) file system, we can not move between two different file
+    // systems.
+    Path archiveDir = HFileArchiveUtil.getStoreArchivePath(conf, region.getRegionInfo(),
+      store.getColumnFamilyDescriptor().getName());
+    FileSystem fs = archiveDir.getFileSystem(conf);
+    if (LOG.isDebugEnabled()) {
+      FileStatus[] toDelete = fs.listStatus(archiveDir);
+      if (toDelete != null && toDelete.length > 0) {
+        LOG.debug("Delete all archived HFiles under {}: {}", archiveDir, Stream.of(toDelete)
+          .map(s -> s.getPath().getName()).collect(Collectors.joining(", ", "[", "]")));
+      }
+    }
+    fs.delete(archiveDir, true);
+  }
+
   private void compact() {
     try {
       region.compact(true);
+      deleteCompactedHFiles();
 
 Review comment:
   We just delete archived files? Why even archive them then? Can we delay the delete at least?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services