You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Gary Helmling <gh...@gmail.com> on 2010/10/26 23:21:13 UTC
Re: Review Request: HBASE-3102: Add Capability to Monitor Long Running
Metrics
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1055/#review1672
-----------------------------------------------------------
Looks pretty good. These look like great metrics to add!
Only issue I see is some odd naming that will happen for attributes used for the "*Size" metrics. I'd consider copying the rest of the implementation from MetricsTimeVaryingRate -> PersistentMetricsTimeVaryingRate and extending MetricsBase directly, to allow tweaking the naming. Thoughts?
A few trailing whitespace issues throughout. Otherwise looks great!
trunk/src/main/java/org/apache/hadoop/hbase/master/metrics/MasterMetrics.java
<http://review.cloudera.org/r/1055/#comment5547>
I think using this type will result in the average split size for the period being reported as "splitSize_avg_time"? The value should be right, the attribute name is just a little weird since it's size not time.
trunk/src/main/java/org/apache/hadoop/hbase/master/metrics/MasterMetrics.java
<http://review.cloudera.org/r/1055/#comment5546>
Drop trailing whitespace here and following sections
trunk/src/main/java/org/apache/hadoop/hbase/metrics/PersistentMetricsTimeVaryingRate.java
<http://review.cloudera.org/r/1055/#comment5548>
Given the above naming of "splitSize_avg_time", maybe consider extending MetricsBase directly and essentially copy-n-paste forking MetricsTimeVaryingRate?
I know it sucks not being able to subclass cleanly here, but then you could at least expose the attributes as:
- name + "_num_ops"
- name + "_avg"
and avoid the naming confusion.
trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
<http://review.cloudera.org/r/1055/#comment5549>
Same slightly weird naming for "compactionSize" and "flushSize" metrics will happen as for "splitSize" in MasterMetrics.
- Gary
On 2010-10-21 18:59:12, Nicolas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.cloudera.org/r/1055/
> -----------------------------------------------------------
>
> (Updated 2010-10-21 18:59:12)
>
>
> Review request for hbase.
>
>
> Summary
> -------
>
> Example: a useful metric to observe track is compaction count + duration. Since compactions are long running and only happen infrequently, the avg/opcount stats are should be reset longer than the default polling period (5 sec). In addition to 'hbase.period', we should allow a different duration after which long-running metrics should expire. This would also fix our existing metrics problem where min/max stats are never reset until the process is restarted/upgraded.
>
> This diff adds the capability for long-running metrics in addition to the infrastructure necessary to log compactions + log splits.
>
>
> This addresses bug HBASE-3102.
> http://issues.apache.org/jira/browse/HBASE-3102
>
>
> Diffs
> -----
>
> trunk/conf/hadoop-metrics.properties 1025790
> trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1025790
> trunk/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 1025790
> trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1025790
> trunk/src/main/java/org/apache/hadoop/hbase/master/metrics/MasterMetrics.java 1025790
> trunk/src/main/java/org/apache/hadoop/hbase/metrics/PersistentMetricsTimeVaryingRate.java PRE-CREATION
> trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1025790
> trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1025790
> trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java 1025790
> trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1025790
> trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java 1025790
> trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 1025790
> trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java 1025790
> trunk/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1025790
>
> Diff: http://review.cloudera.org/r/1055/diff
>
>
> Testing
> -------
>
> mvn clean install
> cluster load test
> ** Note: this was primarily tested on 0.89. Minimal testing on 0.90
>
>
> Thanks,
>
> Nicolas
>
>