You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "dlmarion (via GitHub)" <gi...@apache.org> on 2025/04/16 21:48:20 UTC

[PR] Updated TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND options [accumulo]

dlmarion opened a new pull request, #5485:
URL: https://github.com/apache/accumulo/pull/5485

   This is a follow-on to #5473 that changes the property options to named values instead of using the internal file prefixes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Updated TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND options [accumulo]

Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #5485:
URL: https://github.com/apache/accumulo/pull/5485#discussion_r2049415992


##########
server/base/src/main/java/org/apache/accumulo/server/compaction/FileCompactor.java:
##########
@@ -346,10 +346,21 @@ public CompactionStats call()
       // have compacted, or in the case of cloned tables where one
       // of the tables has compacted the input file but the other
       // has not.
-      String dropCachePrefixProperty =
+      final String dropCachePrefixProperty =
           acuTableConf.get(Property.TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND);
-      final EnumSet<FileTypePrefix> dropCacheFilePrefixes =
-          FileTypePrefix.typesFromList(dropCachePrefixProperty);
+      final EnumSet<FileTypePrefix> dropCacheFileTypes;
+      if (dropCachePrefixProperty.equalsIgnoreCase("ALL")) {
+        dropCacheFileTypes = EnumSet.allOf(FileTypePrefix.class);
+      } else if (dropCachePrefixProperty.equalsIgnoreCase("NON-IMPORT")) {
+        dropCacheFileTypes = EnumSet.of(FileTypePrefix.FLUSH, FileTypePrefix.FULL_COMPACTION,
+            FileTypePrefix.COMPACTION, FileTypePrefix.MERGING_MINOR_COMPACTION);
+      } else if (dropCachePrefixProperty.equalsIgnoreCase("NONE")) {
+        dropCacheFileTypes = EnumSet.noneOf(FileTypePrefix.class);
+      } else {
+        throw new IllegalArgumentException("Invalid value for property "
+            + Property.TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND.getKey()
+            + " expected one of ALL, NONE, or NON-IMPORT");
+      }

Review Comment:
   Created PR #5486 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Updated TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND options [accumulo]

Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #5485:
URL: https://github.com/apache/accumulo/pull/5485#discussion_r2047824225


##########
server/base/src/main/java/org/apache/accumulo/server/fs/FileTypePrefix.java:
##########
@@ -18,27 +18,18 @@
  */
 package org.apache.accumulo.server.fs;
 
-import java.util.EnumSet;
-import java.util.HashSet;
 import java.util.Objects;
-import java.util.Set;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 
 import com.google.common.base.Preconditions;
 
 public enum FileTypePrefix {
 
-  ALL("*"),
-  FLUSH("F"),
+  FLUSH(
+      "F"),

Review Comment:
   I have no idea why the formatter is putting a new line here. I removed it locally, and it put it back.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Updated TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND options [accumulo]

Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #5485:
URL: https://github.com/apache/accumulo/pull/5485#discussion_r2049415992


##########
server/base/src/main/java/org/apache/accumulo/server/compaction/FileCompactor.java:
##########
@@ -346,10 +346,21 @@ public CompactionStats call()
       // have compacted, or in the case of cloned tables where one
       // of the tables has compacted the input file but the other
       // has not.
-      String dropCachePrefixProperty =
+      final String dropCachePrefixProperty =
           acuTableConf.get(Property.TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND);
-      final EnumSet<FileTypePrefix> dropCacheFilePrefixes =
-          FileTypePrefix.typesFromList(dropCachePrefixProperty);
+      final EnumSet<FileTypePrefix> dropCacheFileTypes;
+      if (dropCachePrefixProperty.equalsIgnoreCase("ALL")) {
+        dropCacheFileTypes = EnumSet.allOf(FileTypePrefix.class);
+      } else if (dropCachePrefixProperty.equalsIgnoreCase("NON-IMPORT")) {
+        dropCacheFileTypes = EnumSet.of(FileTypePrefix.FLUSH, FileTypePrefix.FULL_COMPACTION,
+            FileTypePrefix.COMPACTION, FileTypePrefix.MERGING_MINOR_COMPACTION);
+      } else if (dropCachePrefixProperty.equalsIgnoreCase("NONE")) {
+        dropCacheFileTypes = EnumSet.noneOf(FileTypePrefix.class);
+      } else {
+        throw new IllegalArgumentException("Invalid value for property "
+            + Property.TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND.getKey()
+            + " expected one of ALL, NONE, or NON-IMPORT");
+      }

Review Comment:
   Created PR #5486 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Updated TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND options [accumulo]

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #5485:
URL: https://github.com/apache/accumulo/pull/5485#discussion_r2048075896


##########
server/base/src/main/java/org/apache/accumulo/server/fs/FileTypePrefix.java:
##########
@@ -18,27 +18,18 @@
  */
 package org.apache.accumulo.server.fs;
 
-import java.util.EnumSet;
-import java.util.HashSet;
 import java.util.Objects;
-import java.util.Set;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
 
 import com.google.common.base.Preconditions;
 
 public enum FileTypePrefix {
 
-  ALL("*"),
-  FLUSH("F"),
+  FLUSH(
+      "F"),

Review Comment:
   I'm not sure why this happens. There may be some obscure Eclipse formatter config for it, but it can be prevented by adding a comment after, or separating enums with blank lines (perhaps with a javadoc for each, for example).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Updated TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND options [accumulo]

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on PR #5485:
URL: https://github.com/apache/accumulo/pull/5485#issuecomment-2811593485

   @dlmarion I added my code review suggestions in a new commit, b61dc9d5080f7abcf3a45635101db619a1305418
   Feel free to keep it, modify it, drop it, or extract what you like from it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Updated TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND options [accumulo]

Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion merged PR #5485:
URL: https://github.com/apache/accumulo/pull/5485


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Updated TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND options [accumulo]

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #5485:
URL: https://github.com/apache/accumulo/pull/5485#discussion_r2048104173


##########
server/base/src/main/java/org/apache/accumulo/server/compaction/FileCompactor.java:
##########
@@ -346,10 +346,21 @@ public CompactionStats call()
       // have compacted, or in the case of cloned tables where one
       // of the tables has compacted the input file but the other
       // has not.
-      String dropCachePrefixProperty =
+      final String dropCachePrefixProperty =
           acuTableConf.get(Property.TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND);
-      final EnumSet<FileTypePrefix> dropCacheFilePrefixes =
-          FileTypePrefix.typesFromList(dropCachePrefixProperty);
+      final EnumSet<FileTypePrefix> dropCacheFileTypes;
+      if (dropCachePrefixProperty.equalsIgnoreCase("ALL")) {
+        dropCacheFileTypes = EnumSet.allOf(FileTypePrefix.class);
+      } else if (dropCachePrefixProperty.equalsIgnoreCase("NON-IMPORT")) {
+        dropCacheFileTypes = EnumSet.of(FileTypePrefix.FLUSH, FileTypePrefix.FULL_COMPACTION,
+            FileTypePrefix.COMPACTION, FileTypePrefix.MERGING_MINOR_COMPACTION);
+      } else if (dropCachePrefixProperty.equalsIgnoreCase("NONE")) {
+        dropCacheFileTypes = EnumSet.noneOf(FileTypePrefix.class);
+      } else {
+        throw new IllegalArgumentException("Invalid value for property "
+            + Property.TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND.getKey()
+            + " expected one of ALL, NONE, or NON-IMPORT");
+      }

Review Comment:
   I was going to put this in ConfigurationTypeHelper in my code review suggestions commit, but couldn't because the FileTypePrefix is not in the core module. Here is fine, but if this code ends up getting used in more places, it'd be nice to put the conversion in a central place. ConfigurationTypeHelper is generally the place we put conversions from the String properties to the concrete objects the strings represent (like instantiating a class from the property value containing a class name)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Updated TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND options [accumulo]

Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #5485:
URL: https://github.com/apache/accumulo/pull/5485#discussion_r2048932240


##########
server/base/src/main/java/org/apache/accumulo/server/compaction/FileCompactor.java:
##########
@@ -346,10 +346,21 @@ public CompactionStats call()
       // have compacted, or in the case of cloned tables where one
       // of the tables has compacted the input file but the other
       // has not.
-      String dropCachePrefixProperty =
+      final String dropCachePrefixProperty =
           acuTableConf.get(Property.TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND);
-      final EnumSet<FileTypePrefix> dropCacheFilePrefixes =
-          FileTypePrefix.typesFromList(dropCachePrefixProperty);
+      final EnumSet<FileTypePrefix> dropCacheFileTypes;
+      if (dropCachePrefixProperty.equalsIgnoreCase("ALL")) {
+        dropCacheFileTypes = EnumSet.allOf(FileTypePrefix.class);
+      } else if (dropCachePrefixProperty.equalsIgnoreCase("NON-IMPORT")) {
+        dropCacheFileTypes = EnumSet.of(FileTypePrefix.FLUSH, FileTypePrefix.FULL_COMPACTION,
+            FileTypePrefix.COMPACTION, FileTypePrefix.MERGING_MINOR_COMPACTION);
+      } else if (dropCachePrefixProperty.equalsIgnoreCase("NONE")) {
+        dropCacheFileTypes = EnumSet.noneOf(FileTypePrefix.class);
+      } else {
+        throw new IllegalArgumentException("Invalid value for property "
+            + Property.TABLE_COMPACTION_INPUT_DROP_CACHE_BEHIND.getKey()
+            + " expected one of ALL, NONE, or NON-IMPORT");
+      }

Review Comment:
   In the initial commit of `FileTypePrefix` into 2.1 I put the class in server-base. I didn't remember that I had created `FilePrefix` in core in the 3.0 release several years ago. In `main` I merged FileTypePrefix into FilePrefix. In a790064 I went ahead and made the same change so that the 2.1 and main branches have the same FilePrefix class. This allowed me to move the code in FileCompactor to ConfigurationTypeHelper.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org