Major compaction queue monitoring

[Ioakim Perros <im...@gmail.com>]

Hi,

I would like to programmatically major compact a table's regions, but I 
would like to check if major compaction ended (i.e.: compaction queue 
size goes to zero).

I tried to do it through the following code but it failed throwing a 
null pointer exc.

HBaseAdmin admin = new HBaseAdmin(config);
admin.majorCompact("myTable");
admin.close();
HRegionServer hr = new HRegionServer(config);
try {
        if ( ! (hr.getMetrics().compactionQueueSize.get()==0)  ){
             System.out.println( 
hr.getMetrics().compactionQueueSize.get() );
             Thread.sleep(1000);
       }
} catch(InterruptedException ex) {
        Thread.currentThread().interrupt();
}

Does anyone know how to correctly check this?

Thanks in advance,
IP

Re: Major compaction queue monitoring

[Ioakim Perros <im...@gmail.com>]

What I was trying to ensure is that major compactions don't affect data 
read by a subsequent scanner - as I do major compactions periodically 
inside the code in order to free space and improve performance.

Then I found this issue: 
https://issues.apache.org/jira/browse/HBASE-5121 and I was kind of 
relieved to see 0.92.1 in the fix versions.

In any case, thank you very much for responding!
IP

On 09/24/2012 11:47 PM, Jean-Daniel Cryans wrote:
> The way we do it is here is by reading the jmx metrics (in java via
> javax.management.remote) that the region server outputs instead of
> querying directly the region server. What you are doing is kinda
> hacky.
>
> J-D
>
> On Sat, Sep 22, 2012 at 7:58 AM, Ioakim Perros <im...@gmail.com> wrote:
>> Hi,
>>
>> I would like to programmatically major compact a table's regions, but I
>> would like to check if major compaction ended (i.e.: compaction queue size
>> goes to zero).
>>
>> I tried to do it through the following code but it failed throwing a null
>> pointer exc.
>>
>> HBaseAdmin admin = new HBaseAdmin(config);
>> admin.majorCompact("myTable");
>> admin.close();
>> HRegionServer hr = new HRegionServer(config);
>> try {
>>         if ( ! (hr.getMetrics().compactionQueueSize.get()==0)  ){
>>              System.out.println( hr.getMetrics().compactionQueueSize.get() );
>>              Thread.sleep(1000);
>>        }
>> } catch(InterruptedException ex) {
>>         Thread.currentThread().interrupt();
>> }
>>
>> Does anyone know how to correctly check this?
>>
>> Thanks in advance,
>> IP

Re: Major compaction queue monitoring

[Jean-Daniel Cryans <jd...@apache.org>]

The way we do it is here is by reading the jmx metrics (in java via
javax.management.remote) that the region server outputs instead of
querying directly the region server. What you are doing is kinda
hacky.

J-D

On Sat, Sep 22, 2012 at 7:58 AM, Ioakim Perros <im...@gmail.com> wrote:
> Hi,
>
> I would like to programmatically major compact a table's regions, but I
> would like to check if major compaction ended (i.e.: compaction queue size
> goes to zero).
>
> I tried to do it through the following code but it failed throwing a null
> pointer exc.
>
> HBaseAdmin admin = new HBaseAdmin(config);
> admin.majorCompact("myTable");
> admin.close();
> HRegionServer hr = new HRegionServer(config);
> try {
>        if ( ! (hr.getMetrics().compactionQueueSize.get()==0)  ){
>             System.out.println( hr.getMetrics().compactionQueueSize.get() );
>             Thread.sleep(1000);
>       }
> } catch(InterruptedException ex) {
>        Thread.currentThread().interrupt();
> }
>
> Does anyone know how to correctly check this?
>
> Thanks in advance,
> IP