[jira] [Created] (HBASE-26365) Avoid small stripe compaction

["Xiaolin Ha (Jira)" <ji...@apache.org>]

Xiaolin Ha created HBASE-26365:
----------------------------------

             Summary: Avoid small stripe compaction
                 Key: HBASE-26365
                 URL: https://issues.apache.org/jira/browse/HBASE-26365
             Project: HBase
          Issue Type: Improvement
          Components: Compaction
    Affects Versions: 2.0.0, 3.0.0-alpha-1
            Reporter: Xiaolin Ha
            Assignee: Xiaolin Ha


In the StripeCompactPolicy#needsSingleStripeCompaction, it only checks if the file count is up to the min compaction limit.
{code:java}
protected boolean needsSingleStripeCompaction(StripeInformationProvider si) {
  int minFiles = this.config.getStripeCompactMinFiles();
  for (List<HStoreFile> stripe : si.getStripes()) {
    if (stripe.size() >= minFiles) return true;
  }
  return false;
}
{code}
When select files in stripe by ExploringCompactionPolicy#applyCompactionPolicy, there is no min size limit too.
{code:java}
// Sanity checks
if (potentialMatchFiles.size() < minFiles) {
  continue;
}
if (potentialMatchFiles.size() > maxFiles) {
  continue;
}

// Compute the total size of files that will
// have to be read if this set of files is compacted.
long size = getTotalStoreSize(potentialMatchFiles);

// Store the smallest set of files.  This stored set of files will be used
// if it looks like the algorithm is stuck.
if (mightBeStuck && size < smallestSize) {
  smallest = potentialMatchFiles;
  smallestSize = size;
}

if (size > comConf.getMaxCompactSize(mayUseOffPeak)) {
  continue;
}

++opts;
if (size >= comConf.getMinCompactSize()
    && !filesInRatio(potentialMatchFiles, currentRatio)) {
  continue;
}{code}
While we should also limit the min size of compaction for stripes, because small stripe compactions will occupy the compaction queue.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)