You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "keith-turner (via GitHub)" <gi...@apache.org> on 2024/01/09 02:04:55 UTC
[PR] uses table id in compaction prioritization [accumulo]
keith-turner opened a new pull request, #4142:
URL: https://github.com/apache/accumulo/pull/4142
Makes the default compaction planner use the table id when prioritizing compactions. The root table id is always given the highest priority, followed by the metadata table, then user tables.
Fixes #4033
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] uses table id in compaction prioritization [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4142:
URL: https://github.com/apache/accumulo/pull/4142#discussion_r1446102343
##########
core/src/main/java/org/apache/accumulo/core/util/compaction/CompactionJobPrioritizer.java:
##########
@@ -20,40 +20,56 @@
import java.util.Comparator;
+import org.apache.accumulo.core.data.TableId;
+import org.apache.accumulo.core.metadata.schema.Ample;
import org.apache.accumulo.core.spi.compaction.CompactionJob;
import org.apache.accumulo.core.spi.compaction.CompactionKind;
+import com.google.common.base.Preconditions;
+
public class CompactionJobPrioritizer {
public static final Comparator<CompactionJob> JOB_COMPARATOR =
Comparator.comparingInt(CompactionJob::getPriority)
.thenComparingInt(job -> job.getFiles().size()).reversed();
- public static short createPriority(CompactionKind kind, int totalFiles, int compactingFiles) {
+ public static short createPriority(TableId tableId, CompactionKind kind, int totalFiles,
+ int compactingFiles) {
+
+ Preconditions.checkArgument(totalFiles >= 0, "totalFiles is negative %s", totalFiles);
+ Preconditions.checkArgument(compactingFiles >= 0, "compactingFiles is negative %s",
+ compactingFiles);
- int prio = totalFiles + compactingFiles;
+ // This holds the two bits used to encode the priority of the table.
Review Comment:
There are two bits for the table and only three of those fours slots currently being used. Can use the 4th slot for #4133. If this is merged before #4133, may need to open an issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] uses table id in compaction prioritization [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4142:
URL: https://github.com/apache/accumulo/pull/4142#discussion_r1446098462
##########
core/src/main/java/org/apache/accumulo/core/util/compaction/CompactionJobPrioritizer.java:
##########
@@ -20,40 +20,56 @@
import java.util.Comparator;
+import org.apache.accumulo.core.data.TableId;
+import org.apache.accumulo.core.metadata.schema.Ample;
import org.apache.accumulo.core.spi.compaction.CompactionJob;
import org.apache.accumulo.core.spi.compaction.CompactionKind;
+import com.google.common.base.Preconditions;
+
public class CompactionJobPrioritizer {
public static final Comparator<CompactionJob> JOB_COMPARATOR =
Comparator.comparingInt(CompactionJob::getPriority)
.thenComparingInt(job -> job.getFiles().size()).reversed();
- public static short createPriority(CompactionKind kind, int totalFiles, int compactingFiles) {
+ public static short createPriority(TableId tableId, CompactionKind kind, int totalFiles,
+ int compactingFiles) {
+
+ Preconditions.checkArgument(totalFiles >= 0, "totalFiles is negative %s", totalFiles);
+ Preconditions.checkArgument(compactingFiles >= 0, "compactingFiles is negative %s",
+ compactingFiles);
- int prio = totalFiles + compactingFiles;
+ // This holds the two bits used to encode the priority of the table.
Review Comment:
Yeah it could. We could add the table id from that work into this calculation once its merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] uses table id in compaction prioritization [accumulo]
Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #4142:
URL: https://github.com/apache/accumulo/pull/4142#discussion_r1445608594
##########
core/src/main/java/org/apache/accumulo/core/util/compaction/CompactionJobPrioritizer.java:
##########
@@ -20,40 +20,56 @@
import java.util.Comparator;
+import org.apache.accumulo.core.data.TableId;
+import org.apache.accumulo.core.metadata.schema.Ample;
import org.apache.accumulo.core.spi.compaction.CompactionJob;
import org.apache.accumulo.core.spi.compaction.CompactionKind;
+import com.google.common.base.Preconditions;
+
public class CompactionJobPrioritizer {
public static final Comparator<CompactionJob> JOB_COMPARATOR =
Comparator.comparingInt(CompactionJob::getPriority)
.thenComparingInt(job -> job.getFiles().size()).reversed();
- public static short createPriority(CompactionKind kind, int totalFiles, int compactingFiles) {
+ public static short createPriority(TableId tableId, CompactionKind kind, int totalFiles,
+ int compactingFiles) {
+
+ Preconditions.checkArgument(totalFiles >= 0, "totalFiles is negative %s", totalFiles);
+ Preconditions.checkArgument(compactingFiles >= 0, "compactingFiles is negative %s",
+ compactingFiles);
- int prio = totalFiles + compactingFiles;
+ // This holds the two bits used to encode the priority of the table.
Review Comment:
Does the work in #4133 change anything with this compaction priority calculation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] uses table id in compaction prioritization [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner merged PR #4142:
URL: https://github.com/apache/accumulo/pull/4142
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org