You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Tim Sell <tr...@gmail.com> on 2011/05/23 19:39:45 UTC
HBase jmx stats
I am collecting hbase stats from jmx.
Here's an example of the compaction stats from one of the region servers...
compactionQueueSize 0
compactionSizeAvgTime 19349729
compactionSizeMaxTime 566512933
compactionSizeMinTime 0
compactionSizeNumOps 8
compactionTimeAvgTime 83
compactionTimeMaxTime 1300
compactionTimeMinTime 0
compactionTimeNumOps 8
What does NumOps of compactionSize and compactionTime mean? is that
the total number of compactions ever?
Also, having a time suffix for compactionSize seems confusing. It
seems an artifact of this:
protected final PersistentMetricsTimeVaryingRate compactionTime =
new PersistentMetricsTimeVaryingRate("compactionTime", registry);
protected final PersistentMetricsTimeVaryingRate compactionSize =
new PersistentMetricsTimeVaryingRate("compactionSize", registry);
Re: HBase jmx stats
Posted by Ted Dunning <td...@maprtech.com>.
In case anybody wants estimates of medians, Mahout has some easily
extractable code to compute medians and first and third quartiles without
keeping lots of data around. As a side effect, it computes averages and
standard deviations as well.
I don't think that such a small thing as this warrants an entire dependency
and would be happy to extract the code in question if anybody wants this.
Just point me at a JIRA where I should stick it.
On Tue, May 24, 2011 at 10:17 AM, Stack <st...@duboce.net> wrote:
> Tim:
>
> You should get a copy of Lars' book. Then you'd know what the below
> are (smile).
>
> Quoting:
>
> "A commonly used metric in HBase is called time varying rate, which
> not only tracks the number of events, but also how long each event
> took to complete. A TVR exposes four values. [Below] shows the
> values and their meanings.
>
> Value Name Short Description
> Number Operations NumOps The actual number of events since the last
> poll
> Minimum Time MniTime The shortest time reported for an event
> to complete
> Maximum Time MaxTime The longest time reported for an event to
> complete
> Average Time AvgTime The average time for completing
> events...sum of times per event divided by number of events"
>
> Our metrics need a bit of a revamp so file issues on stuff that seems off
> Tim.
>
> St.Ack
>
> On Mon, May 23, 2011 at 10:39 AM, Tim Sell <tr...@gmail.com> wrote:
> > I am collecting hbase stats from jmx.
> >
> > Here's an example of the compaction stats from one of the region
> servers...
> >
> > compactionQueueSize 0
> > compactionSizeAvgTime 19349729
> > compactionSizeMaxTime 566512933
> > compactionSizeMinTime 0
> > compactionSizeNumOps 8
> > compactionTimeAvgTime 83
> > compactionTimeMaxTime 1300
> > compactionTimeMinTime 0
> > compactionTimeNumOps 8
> >
> > What does NumOps of compactionSize and compactionTime mean? is that
> > the total number of compactions ever?
> >
> > Also, having a time suffix for compactionSize seems confusing. It
> > seems an artifact of this:
> >
> > protected final PersistentMetricsTimeVaryingRate compactionTime =
> > new PersistentMetricsTimeVaryingRate("compactionTime", registry);
> > protected final PersistentMetricsTimeVaryingRate compactionSize =
> > new PersistentMetricsTimeVaryingRate("compactionSize", registry);
> >
>
Re: SingleColumnValueFilter
Posted by 梁景明 <fu...@gmail.com>.
FilterList list = new FilterList(FilterList.Operator.MUST_PASS_ALL, fs);
u can define a list like this
在 2011年5月25日 上午10:39， <hm...@tsmc.com>写道：
>
>
> Hi there,
>
> Can I set multiple SingleColumnValueFilter for a qualifier like following
> sample codes?
> The scan result is nothing, so I think whether the filters can combine
> these filter with "OR" instead of " AND".
> Any ideas?
>
>
>
> //=======================================================================================
> Scan s = new Scan();
> FilterList fl = new FilterList();
> SingleColumnValueFilter scv1 = new SingleColumnValueFilter
> (Bytes.toBytes("cf"),
> Bytes.toBytes("q1"),
> CompareFilter.CompareOp.EQUAL,
> new RegexStringComparator("^AA")) ;
>
> SingleColumnValueFilter scv2 = new SingleColumnValueFilter
> (Bytes.toBytes("cf"),
> Bytes.toBytes("q1"),
> CompareFilter.CompareOp.EQUAL,
> new RegexStringComparator("^BB")) ;
>
> SingleColumnValueFilter scv3 = new SingleColumnValueFilter
> (Bytes.toBytes("cf"),
> Bytes.toBytes("q1"),
> CompareFilter.CompareOp.EQUAL,
> new RegexStringComparator("^CC")) ;
>
> fl.addFilter(scv1);
> fl.addFilter(scv2);
> fl.addFilter(scv3);
> s.setFilter(fl);
>
>
> Fleming Chiu(邱宏明)
> Ext: 707-2260
> Be Veg, Go Green, Save the Planet!
>
> ---------------------------------------------------------------------------
> TSMC PROPERTY
> This email communication (and any attachments) is proprietary information
> for the sole use of its
> intended recipient. Any unauthorized review, use or distribution by anyone
> other than the intended
> recipient is strictly prohibited. If you are not the intended recipient,
> please notify the sender by
> replying to this email, and then delete this email and any copies of it
> immediately. Thank you.
>
> ---------------------------------------------------------------------------
>
>
>
>
SingleColumnValueFilter
Posted by hm...@tsmc.com.
Hi there,
Can I set multiple SingleColumnValueFilter for a qualifier like following
sample codes?
The scan result is nothing, so I think whether the filters can combine
these filter with "OR" instead of " AND".
Any ideas?
//=======================================================================================
Scan s = new Scan();
FilterList fl = new FilterList();
SingleColumnValueFilter scv1 = new SingleColumnValueFilter
(Bytes.toBytes("cf"),
Bytes.toBytes("q1"),
CompareFilter.CompareOp.EQUAL,
new RegexStringComparator("^AA")) ;
SingleColumnValueFilter scv2 = new SingleColumnValueFilter
(Bytes.toBytes("cf"),
Bytes.toBytes("q1"),
CompareFilter.CompareOp.EQUAL,
new RegexStringComparator("^BB")) ;
SingleColumnValueFilter scv3 = new SingleColumnValueFilter
(Bytes.toBytes("cf"),
Bytes.toBytes("q1"),
CompareFilter.CompareOp.EQUAL,
new RegexStringComparator("^CC")) ;
fl.addFilter(scv1);
fl.addFilter(scv2);
fl.addFilter(scv3);
s.setFilter(fl);
Fleming Chiu(邱宏明)
Ext: 707-2260
Be Veg, Go Green, Save the Planet!
---------------------------------------------------------------------------
TSMC PROPERTY
This email communication (and any attachments) is proprietary information
for the sole use of its
intended recipient. Any unauthorized review, use or distribution by anyone
other than the intended
recipient is strictly prohibited. If you are not the intended recipient,
please notify the sender by
replying to this email, and then delete this email and any copies of it
immediately. Thank you.
---------------------------------------------------------------------------
Re: HBase jmx stats
Posted by Stack <st...@duboce.net>.
Tim:
You should get a copy of Lars' book. Then you'd know what the below
are (smile).
Quoting:
"A commonly used metric in HBase is called time varying rate, which
not only tracks the number of events, but also how long each event
took to complete. A TVR exposes four values. [Below] shows the
values and their meanings.
Value Name Short Description
Number Operations NumOps The actual number of events since the last poll
Minimum Time MniTime The shortest time reported for an event
to complete
Maximum Time MaxTime The longest time reported for an event to complete
Average Time AvgTime The average time for completing
events...sum of times per event divided by number of events"
Our metrics need a bit of a revamp so file issues on stuff that seems off Tim.
St.Ack
On Mon, May 23, 2011 at 10:39 AM, Tim Sell <tr...@gmail.com> wrote:
> I am collecting hbase stats from jmx.
>
> Here's an example of the compaction stats from one of the region servers...
>
> compactionQueueSize 0
> compactionSizeAvgTime 19349729
> compactionSizeMaxTime 566512933
> compactionSizeMinTime 0
> compactionSizeNumOps 8
> compactionTimeAvgTime 83
> compactionTimeMaxTime 1300
> compactionTimeMinTime 0
> compactionTimeNumOps 8
>
> What does NumOps of compactionSize and compactionTime mean? is that
> the total number of compactions ever?
>
> Also, having a time suffix for compactionSize seems confusing. It
> seems an artifact of this:
>
> protected final PersistentMetricsTimeVaryingRate compactionTime =
> new PersistentMetricsTimeVaryingRate("compactionTime", registry);
> protected final PersistentMetricsTimeVaryingRate compactionSize =
> new PersistentMetricsTimeVaryingRate("compactionSize", registry);
>