[jira] [Commented] (HBASE-14651) Default minimum compaction size is too high

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-14651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14964317#comment-14964317 ] 

Hadoop QA commented on HBASE-14651:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12767495/HBASE-14651-v1.patch
  against master branch at commit c1f0442045f44fcbb3935f9244794929a5d0caea.
  ATTACHMENT ID: 12767495

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1)

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 protoc{color}.  The applied patch does not increase the total number of protoc compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 checkstyle{color}.  The applied patch does not increase the total number of checkstyle errors

    {color:green}+1 findbugs{color}.  The patch does not introduce any  new Findbugs (version 2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection
                  org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16100//testReport/
Release Findbugs (version 2.0.3) 	warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16100//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16100//artifact/patchprocess/checkstyle-aggregate.html

  Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16100//console

This message is automatically generated.

> Default minimum compaction size is too high
> -------------------------------------------
>
>                 Key: HBASE-14651
>                 URL: https://issues.apache.org/jira/browse/HBASE-14651
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>         Attachments: HBASE-14651-v1.patch
>
>
> *hbase.hstore.compaction.min.size* defines minimum selection size which is always eligible for minor compaction (no compaction ratio check is performed on such file selections). Default size is equals to memstore flush size (128MB).  First of all, even this value is too high for some (many) deployments, especially for write intensive, because of  a small sizes of a memstore flushes, and if user increases memstore flush size (they usually set it to at least 256MB), they have no idea how will it impact the overall compaction process efficiency. With 256MB of minimum size to compact, compactor most of the time skips necessary file ratio checks and this will result in increased read/write IO during compactions, because of the unbalanced selections where relatively large files can be mixed with a newly created small store files. I think we should set this default minimum  to 64MB and not to link it to memstore flush size at all.     



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)