Naive Question on HBase Monitoring

[Michael Dagaev <mi...@gmail.com>]

Hi, all

    I am going to test HBase in a small EC2 cluster. What should I monitor ?
First of all, I am interested in stability now (not performance). Obviously,
I should monitor HBase failures, the disk space, the master status and probably
the status of the region servers. What else ?

In order to monitor that stuff, I am going to check the HBase logs (I guess,
I will find the log location in log4j properties) and run tail, du,
jps, ps commands
periodically. Can you recommend any other tool for the monitoring ?

Thank you for your cooperation,
M.

Re: Naive Question on HBase Monitoring

[stack <st...@duboce.net>]

Michael, if you haven't seen this note by Daniel Leffel using hbase up on
ec2, it might be of help:
http://mail-archives.apache.org/mod_mbox/hadoop-hbase-user/200805.mbox/%3C25e5a0c00805072129w3b54599r286940f134c6f235@mail.gmail.com%3E
Its old but basic notions still apply.  He also suggested elsewhere that you
use x-large instances.

Check out the FAQ.  Has how to enable DEBUG, up file descriptors, etc., some
suggested config. if using hadoop 0.18.x.  If you can, up default RAM.  You
can later play with what optimal heap should be.

Logs by default are under ${HBASE_HOME}/logs.  To change this, edit
${HBASE_HOME}/conf/hbase-env.sh and set HBASE_LOGS_DIR.

Watch the master UI for a sense of what the cluster is up to.  Tailing the
master log will give more detail; you'll see 'events' like region splits and
usually if a regionserver is having trouble, its difficulty should show in
the master log.

Ganglia is also useful getting a sense of whats going on cluster-wise but if
you want to see the hbase metrics, you need to run hbase trunk.

St.Ack


On Fri, Nov 28, 2008 at 4:29 AM, Michael Dagaev <mi...@gmail.com>wrote:

> Hi, all
>
>    I am going to test HBase in a small EC2 cluster. What should I monitor ?
> First of all, I am interested in stability now (not performance).
> Obviously,
> I should monitor HBase failures, the disk space, the master status and
> probably
> the status of the region servers. What else ?
>
> In order to monitor that stuff, I am going to check the HBase logs (I
> guess,
> I will find the log location in log4j properties) and run tail, du,
> jps, ps commands
> periodically. Can you recommend any other tool for the monitoring ?
>
> Thank you for your cooperation,
> M.
>