[jira] [Commented] (HBASE-14906) Improvements on FlushLargeStoresPolicy

["Yu Li (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15035442#comment-15035442 ] 

Yu Li commented on HBASE-14906:
-------------------------------

The result of TestPerColumnFamilyFlush#testCompareStoreFileCount shows a promising improvement (further less flush for small cf):

w/o patch:
2015-12-01 22:15:39,749 INFO  [Thread-1] regionserver.TestPerColumnFamilyFlush(637): disable selective flush: f1=>11, f2=>11, f3=>11
2015-12-01 22:15:39,749 INFO  [Thread-1] regionserver.TestPerColumnFamilyFlush(640): enable selective flush: f1=>6, f2=>9, f3=>12

w/ patch:
2015-12-01 22:23:21,649 INFO  [Thread-1] regionserver.TestPerColumnFamilyFlush(634): disable selective flush: f1=>11, f2=>11, f3=>11
2015-12-01 22:23:21,649 INFO  [Thread-1] regionserver.TestPerColumnFamilyFlush(637): enable selective flush: f1=>6, f2=>7, f3=>13

> Improvements on FlushLargeStoresPolicy
> --------------------------------------
>
>                 Key: HBASE-14906
>                 URL: https://issues.apache.org/jira/browse/HBASE-14906
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 2.0.0
>            Reporter: Yu Li
>            Assignee: Yu Li
>         Attachments: HBASE-14906.patch
>
>
> When checking FlushLargeStoragePolicy, found below possible improving points:
> 1. Currently in selectStoresToFlush, we will do the selection no matter how many actual families, which is not necessary for one single family
> 2. Default value for hbase.hregion.percolumnfamilyflush.size.lower.bound could not fit in all cases, and requires user to know details of the implementation to properly set it. We propose to use "hbase.hregion.memstore.flush.size/column_family_number" instead:
> {noformat}
>   <property>
>     <name>hbase.hregion.percolumnfamilyflush.size.lower.bound</name>
>     <value>16777216</value>
>     <description>
>     If FlushLargeStoresPolicy is used and there are multiple column families,
>     then every time that we hit the total memstore limit, we find out all the
>     column families whose memstores exceed a "lower bound" and only flush them
>     while retaining the others in memory. The "lower bound" will be
>     "hbase.hregion.memstore.flush.size / column_family_number" by default
>     unless value of this property is larger than that. If none of the families
>     have their memstore size more than lower bound, all the memstores will be
>     flushed (just as usual).
>     </description>
>   </property>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)