You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Neal Young (JIRA)" <ji...@apache.org> on 2016/10/18 22:34:58 UTC
[jira] [Updated] (HBASE-16876) RatioBasedCompactionPolicy ignores
mayBeStuck
[ https://issues.apache.org/jira/browse/HBASE-16876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neal Young updated HBASE-16876:
-------------------------------
Description:
I'm a newbie so may not be reporting this bug correctly. The bug currently shows in lines 181 - 190 here : http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.html#181 .
{code:title=Bar.java|borderStyle=solid}
176 while (countOfFiles - start >= comConf.getMinFilesToCompact() &&
177 fileSizes[start] > Math.max(comConf.getMinCompactSize(),
178 (long) (sumSize[start + 1] * ratio))) {
179 ++start;
180 }
181 if (start < countOfFiles) {
182 LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
183 + " files from " + countOfFiles + " candidates");
184 } else if (mayBeStuck) {
185 // We may be stuck. Compact the latest files if we can.
186 int filesToLeave = candidates.size() - comConf.getMinFilesToCompact();
187 if (filesToLeave >= 0) {
188 start = filesToLeave;
189 }
190 }
{code}
On reaching line 176, start = 0. When comConf.getMinFilesToCompact() is at least 2 (as occurs in the default), the while loop is guaranteed to terminate with start < countOfFiles. Hence, the else clause starting at line 184 never executes, regardless of the value of mayBeStuck.
Perhaps the following code would be better, but I'm not sure:
{code:title=Bar.java|borderStyle=solid}
while (start < countOfFiles
&& fileSizes[start] > Math.max(comConf.getMinCompactSize(),
(long) (sumSize[start + 1] * ratio))) {
++start;
}
if (countOfFiles - start < comConf.getMinFilesToCompact()) {
if (mayBeStuck && countOfFiles >= comConf.getMinFilesToCompact()) {
start = countOfFiles - comConf.getMinFilesToCompact();
} else {
start = countOfFiles;
}
}
if (countOfFiles - start > 0) {
LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
+ " files from " + countOfFiles + " candidates");
}
{code}
was:
I'm a newbie so may not be reporting this bug correctly. The bug currently shows in lines 181 - 190 here : http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.html#181 .
{code:title=Bar.java|borderStyle=solid}
176 while (countOfFiles - start >= comConf.getMinFilesToCompact() &&
177 fileSizes[start] > Math.max(comConf.getMinCompactSize(),
178 (long) (sumSize[start + 1] * ratio))) {
179 ++start;
180 }
181 if (start < countOfFiles) {
182 LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
183 + " files from " + countOfFiles + " candidates");
184 } else if (mayBeStuck) {
185 // We may be stuck. Compact the latest files if we can.
186 int filesToLeave = candidates.size() - comConf.getMinFilesToCompact();
187 if (filesToLeave >= 0) {
188 start = filesToLeave;
189 }
190 }
{code}
On reaching line 176, start = 0. When comConf.getMinFilesToCompact() is at least 2 (as occurs in the default), the while loop is guaranteed to terminate with start < countOfFiles. Hence, the else clause starting at line 184 never executes, regardless of the value of mayBeStuck.
Perhaps the following code would be better, but I'm not sure:
{code:title=Bar.java|borderStyle=solid}
while (start < countOfFiles
&& fileSizes[start] > Math.max(comConf.getMinCompactSize(),
(long) (sumSize[start + 1] * ratio))) {
++start;
}
if (start < countOfFiles) {
if (countOfFiles - start >= comConf.getMinFilesToCompact()) {
LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
+ " files from " + countOfFiles + " candidates");
} else {
start = countOfFiles;
}
} else if (mayBeStuck) {
// We may be stuck. Compact the latest files if we can.
int filesToLeave = candidates.size() - comConf.getMinFilesToCompact();
if (filesToLeave >= 0) {
start = filesToLeave;
}
}
{code}
> RatioBasedCompactionPolicy ignores mayBeStuck
> ---------------------------------------------
>
> Key: HBASE-16876
> URL: https://issues.apache.org/jira/browse/HBASE-16876
> Project: HBase
> Issue Type: Bug
> Components: Compaction
> Reporter: Neal Young
> Labels: newbie
>
> I'm a newbie so may not be reporting this bug correctly. The bug currently shows in lines 181 - 190 here : http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.html#181 .
> {code:title=Bar.java|borderStyle=solid}
> 176 while (countOfFiles - start >= comConf.getMinFilesToCompact() &&
> 177 fileSizes[start] > Math.max(comConf.getMinCompactSize(),
> 178 (long) (sumSize[start + 1] * ratio))) {
> 179 ++start;
> 180 }
> 181 if (start < countOfFiles) {
> 182 LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
> 183 + " files from " + countOfFiles + " candidates");
> 184 } else if (mayBeStuck) {
> 185 // We may be stuck. Compact the latest files if we can.
> 186 int filesToLeave = candidates.size() - comConf.getMinFilesToCompact();
> 187 if (filesToLeave >= 0) {
> 188 start = filesToLeave;
> 189 }
> 190 }
> {code}
> On reaching line 176, start = 0. When comConf.getMinFilesToCompact() is at least 2 (as occurs in the default), the while loop is guaranteed to terminate with start < countOfFiles. Hence, the else clause starting at line 184 never executes, regardless of the value of mayBeStuck.
> Perhaps the following code would be better, but I'm not sure:
> {code:title=Bar.java|borderStyle=solid}
> while (start < countOfFiles
> && fileSizes[start] > Math.max(comConf.getMinCompactSize(),
> (long) (sumSize[start + 1] * ratio))) {
> ++start;
> }
> if (countOfFiles - start < comConf.getMinFilesToCompact()) {
> if (mayBeStuck && countOfFiles >= comConf.getMinFilesToCompact()) {
> start = countOfFiles - comConf.getMinFilesToCompact();
> } else {
> start = countOfFiles;
> }
> }
> if (countOfFiles - start > 0) {
> LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
> + " files from " + countOfFiles + " candidates");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)