Experience with HBASE-8283 and lots of small hfile

[Rahul Ravindran <ra...@yahoo.com>]

Hi, 
  We are currently on 0.94.2(CDH 4.2.1) and would likely upgrade to 0.94.15 (CDH 4.6) primarily to use the above fix. We have turned off automatic major compactions. We load data into an hbase table every 2 minutes. Currently, we are not using bulk load since it created compaction issues. We noticed HBASE-8283 and could move to use this. Any gotchas on using this in production? Since, we could create a new HFile every 2 minutes, we would soon have a scenario where we would have a lot of hfiles. Would triggering a non major-compaction (using https://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#compact(byte[])) periodically be a reasonable compromise along with enabling Hbase-8283


Thanks,
~Rahul.

Re: Experience with HBASE-8283 and lots of small hfile

[宾莉金 <bi...@gmail.com>]

Hi Rahul ,  you may set hbase.server.thread.wakefrequency.multiplier to a
small number, default 1000. So the CompactionChecker will run more often.


2014-04-04 11:42 GMT+08:00 Rahul Ravindran <ra...@yahoo.com>:

> Hi,
>   We are currently on 0.94.2(CDH 4.2.1) and would likely upgrade to
> 0.94.15 (CDH 4.6) primarily to use the above fix. We have turned off
> automatic major compactions. We load data into an hbase table every 2
> minutes. Currently, we are not using bulk load since it created compaction
> issues. We noticed HBASE-8283 and could move to use this. Any gotchas on
> using this in production? Since, we could create a new HFile every 2
> minutes, we would soon have a scenario where we would have a lot of hfiles.
> Would triggering a non major-compaction (using
> https://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#compact(byte[]))
> periodically be a reasonable compromise along with enabling Hbase-8283
>
>
> Thanks,
> ~Rahul.




-- 
*Best Regards,*
 lijin bin