Configs for Hbase compaction switch...

[Vidhyashankar Venkataraman <vi...@yahoo-inc.com>]

I want to run some experiments with the major compactions turned off..

There are two parameters that I can see both of which have to be modified to turn major compactions off (am I right here?).
hbase.hstore.compactionThreshold and hbase.hregion.majorcompaction...

But the config file's comments say that setting compactionThreshold to a high value may lead to memory issues while compaction (because an entire Hfile will have to be logged)..


Another related parameter that I found was:
hbase.regionserver.thread.splitcompactcheckfrequency

The default value is 20000... Does this mean the check happens once every 20000 seconds?

Thank you
Vidhya

Re: Configs for Hbase compaction switch...

[Jean-Daniel Cryans <jd...@apache.org>]

> I want to run some experiments with the major compactions turned off..
>
> There are two parameters that I can see both of which have to be modified to turn major compactions off (am I right here?).
> hbase.hstore.compactionThreshold and hbase.hregion.majorcompaction...
>
> But the config file's comments say that setting compactionThreshold to a high value may lead to memory issues while compaction (because an entire Hfile will have to be logged)..

compactionThreshold is only used for minor compactions, since major
compactions always compact every files together. So setting the other
conf to a very high number will basically disable major compactions. I
trunk, setting it to 0 disables them. See
https://issues.apache.org/jira/browse/HBASE-2559

>
>
> Another related parameter that I found was:
> hbase.regionserver.thread.splitcompactcheckfrequency
>
> The default value is 20000... Does this mean the check happens once every 20000 seconds?

All the time-related configurations are in ms, so that's 20 secs.

J-D