You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Richard Tang <tr...@gmail.com> on 2012/08/09 12:43:10 UTC
is there anyway to turn off compaction in hbase
I am using hbase 0.92.1 and want to disable compaction (both major and
minor compactions) for a period of time. is that configurable in hbase?
Re: is there anyway to turn off compaction in hbase
Posted by yonghu <yo...@gmail.com>.
Harsh is right. You find the wrong place.
regards!
Yong
On Sun, Aug 12, 2012 at 1:40 PM, Harsh J <ha...@cloudera.com> wrote:
> Richard,
>
> The property disables major compactions from happening automatically.
> However, if you choose to do this, you should ensure you have a cron
> job that does trigger major_compact on all tables - for compaction is
> a necessary thing, but you just do not want it to happen at any time
> it likes to.
>
> Also, all properties to be overriden must go to
> $HBASE_HOME/conf/hbase-site.xml, and you do not need to edit source
> files and recompile for every config change you want to do.
>
> On Thu, Aug 9, 2012 at 5:37 PM, Richard Tang <tr...@gmail.com> wrote:
>> That is right. I have tried following properties, in conf/hbase-site.xml.
>> It looks it works to disable both major and minor (in some sense)
>> compactions.
>> Note these properties are defined in src/main/resources/hbase-default.xml,
>> but putting the property there without recompiling the src, it will not
>> work.
>>
>> <property>
>> <name>hbase.hstore.compactionThreshold</name>
>> <value>100</value>
>> <description>
>> If more than this number of HStoreFiles in any one HStore
>> (one HStoreFile is written per flush of memstore) then a compaction
>> is run to rewrite all HStoreFiles files as one. Larger numbers
>> put off compaction but when it runs, it takes longer to complete.
>> </description>
>> </property>
>>
>> <property>
>> <name>hbase.hstore.blockingStoreFiles</name>
>> <value>7</value>
>> <description>
>> If more than this number of StoreFiles in any one Store
>> (one StoreFile is written per flush of MemStore) then updates are
>> blocked for this HRegion until a compaction is completed, or
>> until hbase.hstore.blockingWaitTime has been exceeded.
>> </description>
>> </property>
>>
>> <property>
>> <name>hbase.hstore.blockingWaitTime</name>
>> <value>90000</value>
>> <description>
>> The time an HRegion will block updates for after hitting the StoreFile
>> limit defined by hbase.hstore.blockingStoreFiles.
>> After this time has elapsed, the HRegion will stop blocking updates even
>> if a compaction has not been completed. Default: 90 seconds.
>> </description>
>> </property>
>>
>> <property>
>> <name>hbase.hstore.compaction.max</name>
>> <value>100</value>
>> <description>Max number of HStoreFiles to compact per 'minor' compaction.
>> </description>
>> </property>
>>
>> <property>
>> <name>hbase.hregion.majorcompaction</name>
>> <value>0</value>
>> <description>The time (in miliseconds) between 'major' compactions of all
>> HStoreFiles in a region. Default: 1 day.
>> Set to 0 to disable automated major compactions.
>> </description>
>> </property>
>>
>> On Thu, Aug 9, 2012 at 7:03 AM, henry.kim <he...@gmail.com> wrote:
>>
>>> yes, there is a property in hbase-site.xml
>>>
>>> check the property which is named 'hbase.hregion.majorcompaction'
>>>
>>> if you set it to value 0, there will be no automatic compactions.
>>>
>>> 2012. 8. 9., 오후 7:43, Richard Tang <tr...@gmail.com> 작성:
>>>
>>> > I am using hbase 0.92.1 and want to disable compaction (both major and
>>> > minor compactions) for a period of time. is that configurable in hbase?
>>>
>>>
>
>
>
> --
> Harsh J
Re: is there anyway to turn off compaction in hbase
Posted by Harsh J <ha...@cloudera.com>.
Richard,
The property disables major compactions from happening automatically.
However, if you choose to do this, you should ensure you have a cron
job that does trigger major_compact on all tables - for compaction is
a necessary thing, but you just do not want it to happen at any time
it likes to.
Also, all properties to be overriden must go to
$HBASE_HOME/conf/hbase-site.xml, and you do not need to edit source
files and recompile for every config change you want to do.
On Thu, Aug 9, 2012 at 5:37 PM, Richard Tang <tr...@gmail.com> wrote:
> That is right. I have tried following properties, in conf/hbase-site.xml.
> It looks it works to disable both major and minor (in some sense)
> compactions.
> Note these properties are defined in src/main/resources/hbase-default.xml,
> but putting the property there without recompiling the src, it will not
> work.
>
> <property>
> <name>hbase.hstore.compactionThreshold</name>
> <value>100</value>
> <description>
> If more than this number of HStoreFiles in any one HStore
> (one HStoreFile is written per flush of memstore) then a compaction
> is run to rewrite all HStoreFiles files as one. Larger numbers
> put off compaction but when it runs, it takes longer to complete.
> </description>
> </property>
>
> <property>
> <name>hbase.hstore.blockingStoreFiles</name>
> <value>7</value>
> <description>
> If more than this number of StoreFiles in any one Store
> (one StoreFile is written per flush of MemStore) then updates are
> blocked for this HRegion until a compaction is completed, or
> until hbase.hstore.blockingWaitTime has been exceeded.
> </description>
> </property>
>
> <property>
> <name>hbase.hstore.blockingWaitTime</name>
> <value>90000</value>
> <description>
> The time an HRegion will block updates for after hitting the StoreFile
> limit defined by hbase.hstore.blockingStoreFiles.
> After this time has elapsed, the HRegion will stop blocking updates even
> if a compaction has not been completed. Default: 90 seconds.
> </description>
> </property>
>
> <property>
> <name>hbase.hstore.compaction.max</name>
> <value>100</value>
> <description>Max number of HStoreFiles to compact per 'minor' compaction.
> </description>
> </property>
>
> <property>
> <name>hbase.hregion.majorcompaction</name>
> <value>0</value>
> <description>The time (in miliseconds) between 'major' compactions of all
> HStoreFiles in a region. Default: 1 day.
> Set to 0 to disable automated major compactions.
> </description>
> </property>
>
> On Thu, Aug 9, 2012 at 7:03 AM, henry.kim <he...@gmail.com> wrote:
>
>> yes, there is a property in hbase-site.xml
>>
>> check the property which is named 'hbase.hregion.majorcompaction'
>>
>> if you set it to value 0, there will be no automatic compactions.
>>
>> 2012. 8. 9., 오후 7:43, Richard Tang <tr...@gmail.com> 작성:
>>
>> > I am using hbase 0.92.1 and want to disable compaction (both major and
>> > minor compactions) for a period of time. is that configurable in hbase?
>>
>>
--
Harsh J
Re: is there anyway to turn off compaction in hbase
Posted by Richard Tang <tr...@gmail.com>.
That is right. I have tried following properties, in conf/hbase-site.xml.
It looks it works to disable both major and minor (in some sense)
compactions.
Note these properties are defined in src/main/resources/hbase-default.xml,
but putting the property there without recompiling the src, it will not
work.
<property>
<name>hbase.hstore.compactionThreshold</name>
<value>100</value>
<description>
If more than this number of HStoreFiles in any one HStore
(one HStoreFile is written per flush of memstore) then a compaction
is run to rewrite all HStoreFiles files as one. Larger numbers
put off compaction but when it runs, it takes longer to complete.
</description>
</property>
<property>
<name>hbase.hstore.blockingStoreFiles</name>
<value>7</value>
<description>
If more than this number of StoreFiles in any one Store
(one StoreFile is written per flush of MemStore) then updates are
blocked for this HRegion until a compaction is completed, or
until hbase.hstore.blockingWaitTime has been exceeded.
</description>
</property>
<property>
<name>hbase.hstore.blockingWaitTime</name>
<value>90000</value>
<description>
The time an HRegion will block updates for after hitting the StoreFile
limit defined by hbase.hstore.blockingStoreFiles.
After this time has elapsed, the HRegion will stop blocking updates even
if a compaction has not been completed. Default: 90 seconds.
</description>
</property>
<property>
<name>hbase.hstore.compaction.max</name>
<value>100</value>
<description>Max number of HStoreFiles to compact per 'minor' compaction.
</description>
</property>
<property>
<name>hbase.hregion.majorcompaction</name>
<value>0</value>
<description>The time (in miliseconds) between 'major' compactions of all
HStoreFiles in a region. Default: 1 day.
Set to 0 to disable automated major compactions.
</description>
</property>
On Thu, Aug 9, 2012 at 7:03 AM, henry.kim <he...@gmail.com> wrote:
> yes, there is a property in hbase-site.xml
>
> check the property which is named 'hbase.hregion.majorcompaction'
>
> if you set it to value 0, there will be no automatic compactions.
>
> 2012. 8. 9., 오후 7:43, Richard Tang <tr...@gmail.com> 작성:
>
> > I am using hbase 0.92.1 and want to disable compaction (both major and
> > minor compactions) for a period of time. is that configurable in hbase?
>
>
Re: is there anyway to turn off compaction in hbase
Posted by "henry.kim" <he...@gmail.com>.
yes, there is a property in hbase-site.xml
check the property which is named 'hbase.hregion.majorcompaction'
if you set it to value 0, there will be no automatic compactions.
2012. 8. 9., 오후 7:43, Richard Tang <tr...@gmail.com> 작성:
> I am using hbase 0.92.1 and want to disable compaction (both major and
> minor compactions) for a period of time. is that configurable in hbase?