You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Edward Capriolo <ed...@gmail.com> on 2009/12/08 15:52:54 UTC
JMX Metrics with HBASE
List,
I am interested in pulling the HBASE metrics with JMX. I notice one quick thing.
Like hadoop-env hbase.env should have environment variables that allow
setting options per daemon.
Also I have noticed that even though I start a regionserver with JMX
enabled, the information that appears in the HBase Web Interface
is not found in JMX.
request=0.0, regions=4, stores=7, storefiles=9, storefileIndexSize=0,
memstoreSize=0, usedHeap=36, maxHeap=997, blockCacheSize=1719352,
blockCacheFree=207563320, blockCacheCount=2, blockCacheHitRatio=83
The Zookeeper JMX is present when I start the zookeeper component, any
ideas? O can I just get this information with the hbase client?
Edward
Re: JMX Metrics with HBASE
Posted by Gary Helmling <gh...@gmail.com>.
Ah, yes the JMX support was just added in the 0.20.1 release. So you'll
need to use that or later -- 0.20.2 is current so would be recommended -- to
make use of it.
On Tue, Dec 8, 2009 at 11:07 AM, Edward Capriolo <ed...@gmail.com>wrote:
> On Tue, Dec 8, 2009 at 10:22 AM, Gary Helmling <gh...@gmail.com>
> wrote:
> > Hi Edward,
> >
> > If you look at the bin/hbase shell script, you'll see that it adds
> separate
> > env variables to the master, regionserver, thrift and zookeeper processes
> > when running -- respectively:
> >
> > HBASE_MASTER_OPTS
> > HBASE_REGIONSERVER_OPTS
> > HBASE_THRIFT_OPTS
> > HBASE_ZOOKEEPER_OPTS
> >
> > So you just need to add lines for any of these you choose to
> > conf/hbase-env.sh. We should probably add commented out placeholders for
> > each of these in the default hbase-env.sh script with some explanation,
> > though.
> >
> > The equivalent stats to the web interface are exported as the following
> > MBeans:
> >
> > Master:
> > * hadoop:service=Master,name=MasterStatistics
> > * cluster_requests
> > * hadoop:service=HBase,name=RPCStatistics-<portnum>
> > * RpcProcessingTimeAvgTime
> > * RpcProcessingTimeMaxTime
> > * RpcProcessingTimeMinTime
> > * RpcProcessingTimeNumOps
> > * RpcQueueTimeAvgTime
> > * RpcQueueTimeMaxTime
> > * RpcQueueTimeMinTime
> > * RpcQueueTimeNumOps
> >
> > RegionServer:
> > * hadoop:service=RegionServer,name=RegionServerStatistics
> > * atomicIncrementTimeAvgTime
> > * atomicIncrementTimeMaxTime
> > * atomicIncrementTimeMinTime
> > * atomicIncrementTimeNumOps
> > * blockCacheCount
> > * blockCacheFree
> > * blockCacheHitRatio
> > * blockCacheSize
> > * memstoreSizeMB
> > * regions
> > * storefileIndexSizeMB
> > * storefiles
> > * stores
> > * hadoop:service=HBase,name=RPCStatistics-<portnum>
> > * RpcProcessingTimeAvgTime
> > * RpcProcessingTimeMaxTime
> > * RpcProcessingTimeMinTime
> > * RpcProcessingTimeNumOps
> > * RpcQueueTimeAvgTime
> > * RpcQueueTimeMaxTime
> > * RpcQueueTimeMinTime
> > * RpcQueueTimeNumOps
> >
> >
> > The heap usage stats (and many others) are available under the java.lang:
> > domain.
> >
> > If you haven't already, please see the JMX setup instructions:
> > http://hadoop.apache.org/hbase/docs/r0.20.2/metrics.html
> >
> > And if you've already read through that and are running into
> difficulties,
> > let us know what kinds of problems/errors you're seeing.
> >
> > Thanks,
> > Gary
> >
> >
> > On Tue, Dec 8, 2009 at 9:52 AM, Edward Capriolo <edlinuxguru@gmail.com
> >wrote:
> >
> >> List,
> >>
> >> I am interested in pulling the HBASE metrics with JMX. I notice one
> quick
> >> thing.
> >>
> >> Like hadoop-env hbase.env should have environment variables that allow
> >> setting options per daemon.
> >>
> >> Also I have noticed that even though I start a regionserver with JMX
> >> enabled, the information that appears in the HBase Web Interface
> >> is not found in JMX.
> >>
> >> request=0.0, regions=4, stores=7, storefiles=9,
> >> storefileIndexSize=0,
> >> memstoreSize=0, usedHeap=36, maxHeap=997, blockCacheSize=1719352,
> >> blockCacheFree=207563320, blockCacheCount=2, blockCacheHitRatio=83
> >>
> >> The Zookeeper JMX is present when I start the zookeeper component, any
> >> ideas? O can I just get this information with the hbase client?
> >>
> >> Edward
> >>
> >
>
> I think the problem may be that I am using the version from the
> cloudera testing repo and it predates the JMX support? Or has it been
> around for some time.
>
> Thanks for the great info.
>
> Edward
>
Re: JMX Metrics with HBASE
Posted by Edward Capriolo <ed...@gmail.com>.
On Tue, Dec 8, 2009 at 10:22 AM, Gary Helmling <gh...@gmail.com> wrote:
> Hi Edward,
>
> If you look at the bin/hbase shell script, you'll see that it adds separate
> env variables to the master, regionserver, thrift and zookeeper processes
> when running -- respectively:
>
> HBASE_MASTER_OPTS
> HBASE_REGIONSERVER_OPTS
> HBASE_THRIFT_OPTS
> HBASE_ZOOKEEPER_OPTS
>
> So you just need to add lines for any of these you choose to
> conf/hbase-env.sh. We should probably add commented out placeholders for
> each of these in the default hbase-env.sh script with some explanation,
> though.
>
> The equivalent stats to the web interface are exported as the following
> MBeans:
>
> Master:
> * hadoop:service=Master,name=MasterStatistics
> * cluster_requests
> * hadoop:service=HBase,name=RPCStatistics-<portnum>
> * RpcProcessingTimeAvgTime
> * RpcProcessingTimeMaxTime
> * RpcProcessingTimeMinTime
> * RpcProcessingTimeNumOps
> * RpcQueueTimeAvgTime
> * RpcQueueTimeMaxTime
> * RpcQueueTimeMinTime
> * RpcQueueTimeNumOps
>
> RegionServer:
> * hadoop:service=RegionServer,name=RegionServerStatistics
> * atomicIncrementTimeAvgTime
> * atomicIncrementTimeMaxTime
> * atomicIncrementTimeMinTime
> * atomicIncrementTimeNumOps
> * blockCacheCount
> * blockCacheFree
> * blockCacheHitRatio
> * blockCacheSize
> * memstoreSizeMB
> * regions
> * storefileIndexSizeMB
> * storefiles
> * stores
> * hadoop:service=HBase,name=RPCStatistics-<portnum>
> * RpcProcessingTimeAvgTime
> * RpcProcessingTimeMaxTime
> * RpcProcessingTimeMinTime
> * RpcProcessingTimeNumOps
> * RpcQueueTimeAvgTime
> * RpcQueueTimeMaxTime
> * RpcQueueTimeMinTime
> * RpcQueueTimeNumOps
>
>
> The heap usage stats (and many others) are available under the java.lang:
> domain.
>
> If you haven't already, please see the JMX setup instructions:
> http://hadoop.apache.org/hbase/docs/r0.20.2/metrics.html
>
> And if you've already read through that and are running into difficulties,
> let us know what kinds of problems/errors you're seeing.
>
> Thanks,
> Gary
>
>
> On Tue, Dec 8, 2009 at 9:52 AM, Edward Capriolo <ed...@gmail.com>wrote:
>
>> List,
>>
>> I am interested in pulling the HBASE metrics with JMX. I notice one quick
>> thing.
>>
>> Like hadoop-env hbase.env should have environment variables that allow
>> setting options per daemon.
>>
>> Also I have noticed that even though I start a regionserver with JMX
>> enabled, the information that appears in the HBase Web Interface
>> is not found in JMX.
>>
>> request=0.0, regions=4, stores=7, storefiles=9,
>> storefileIndexSize=0,
>> memstoreSize=0, usedHeap=36, maxHeap=997, blockCacheSize=1719352,
>> blockCacheFree=207563320, blockCacheCount=2, blockCacheHitRatio=83
>>
>> The Zookeeper JMX is present when I start the zookeeper component, any
>> ideas? O can I just get this information with the hbase client?
>>
>> Edward
>>
>
I think the problem may be that I am using the version from the
cloudera testing repo and it predates the JMX support? Or has it been
around for some time.
Thanks for the great info.
Edward
Re: JMX Metrics with HBASE
Posted by Gary Helmling <gh...@gmail.com>.
Hi Edward,
If you look at the bin/hbase shell script, you'll see that it adds separate
env variables to the master, regionserver, thrift and zookeeper processes
when running -- respectively:
HBASE_MASTER_OPTS
HBASE_REGIONSERVER_OPTS
HBASE_THRIFT_OPTS
HBASE_ZOOKEEPER_OPTS
So you just need to add lines for any of these you choose to
conf/hbase-env.sh. We should probably add commented out placeholders for
each of these in the default hbase-env.sh script with some explanation,
though.
The equivalent stats to the web interface are exported as the following
MBeans:
Master:
* hadoop:service=Master,name=MasterStatistics
* cluster_requests
* hadoop:service=HBase,name=RPCStatistics-<portnum>
* RpcProcessingTimeAvgTime
* RpcProcessingTimeMaxTime
* RpcProcessingTimeMinTime
* RpcProcessingTimeNumOps
* RpcQueueTimeAvgTime
* RpcQueueTimeMaxTime
* RpcQueueTimeMinTime
* RpcQueueTimeNumOps
RegionServer:
* hadoop:service=RegionServer,name=RegionServerStatistics
* atomicIncrementTimeAvgTime
* atomicIncrementTimeMaxTime
* atomicIncrementTimeMinTime
* atomicIncrementTimeNumOps
* blockCacheCount
* blockCacheFree
* blockCacheHitRatio
* blockCacheSize
* memstoreSizeMB
* regions
* storefileIndexSizeMB
* storefiles
* stores
* hadoop:service=HBase,name=RPCStatistics-<portnum>
* RpcProcessingTimeAvgTime
* RpcProcessingTimeMaxTime
* RpcProcessingTimeMinTime
* RpcProcessingTimeNumOps
* RpcQueueTimeAvgTime
* RpcQueueTimeMaxTime
* RpcQueueTimeMinTime
* RpcQueueTimeNumOps
The heap usage stats (and many others) are available under the java.lang:
domain.
If you haven't already, please see the JMX setup instructions:
http://hadoop.apache.org/hbase/docs/r0.20.2/metrics.html
And if you've already read through that and are running into difficulties,
let us know what kinds of problems/errors you're seeing.
Thanks,
Gary
On Tue, Dec 8, 2009 at 9:52 AM, Edward Capriolo <ed...@gmail.com>wrote:
> List,
>
> I am interested in pulling the HBASE metrics with JMX. I notice one quick
> thing.
>
> Like hadoop-env hbase.env should have environment variables that allow
> setting options per daemon.
>
> Also I have noticed that even though I start a regionserver with JMX
> enabled, the information that appears in the HBase Web Interface
> is not found in JMX.
>
> request=0.0, regions=4, stores=7, storefiles=9,
> storefileIndexSize=0,
> memstoreSize=0, usedHeap=36, maxHeap=997, blockCacheSize=1719352,
> blockCacheFree=207563320, blockCacheCount=2, blockCacheHitRatio=83
>
> The Zookeeper JMX is present when I start the zookeeper component, any
> ideas? O can I just get this information with the hbase client?
>
> Edward
>