Re: Review Request 34447: HIVE-10761 : Create codahale-based metrics system for Hive

[Xuefu Zhang <xz...@cloudera.com>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34447/#review86228
-----------------------------------------------------------

Ship it!



common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/CodahaleMetrics.java
<https://reviews.apache.org/r/34447/#comment138147>

    Nit: It's good to close bw. One problem with this is that if bw=null, while new OutputStreamWriter() is successful, then this new OSW will not get closed. Very unlikely though.


- Xuefu Zhang


On May 31, 2015, 5:19 a.m., Szehon Ho wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34447/
> -----------------------------------------------------------
> 
> (Updated May 31, 2015, 5:19 a.m.)
> 
> 
> Review request for hive, Chao Sun, Jimmy Xiang, and Xuefu Zhang.
> 
> 
> Bugs: HIVE-10761
>     https://issues.apache.org/jira/browse/HIVE-10761
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> See JIRA for the motivation.  Summary: There is an existing metric system that uses some custom model and hooked up to JMX reporting, codahale-based metrics system will be desirable for standard model and reporting.
> 
> This adds a codahale-based metrics system to HiveServer2 and HiveMetastore.  Metrics implementation is now internally pluggable, and the existing Metrics system can be re-enabled by configuration if desired for backward-compatibility.
> 
> Following metrics are supported by Metrics system:
> 1.  JVMPauseMonitor (used to call Hadoop's internal implementation, now forked off to integrate with Metrics system)
> 2.  HMS API calls
> 3.  Standard JVM metrics (only for new implementation, as its free with codahale).
> 
> The following metrics reporting are supported by new system (configuration exposed)
> 1.  JMX
> 2.  CONSOLE
> 3.  JSON_FILE (periodic file of metrics that gets overwritten).
> 
> A goal is to add a webserver that exposes the JSON metrics, but this will defer to a later implementation.
> 
> 
> Diffs
> -----
> 
>   common/pom.xml a615c1e 
>   common/src/java/org/apache/hadoop/hive/common/JvmPauseMonitor.java PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/metrics/LegacyMetrics.java PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java 01c9d1d 
>   common/src/java/org/apache/hadoop/hive/common/metrics/common/Metrics.java PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/metrics/common/MetricsFactory.java PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/CodahaleMetrics.java PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/MetricsReporting.java PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 49b8f97 
>   common/src/test/org/apache/hadoop/hive/common/metrics/TestLegacyMetrics.java PRE-CREATION 
>   common/src/test/org/apache/hadoop/hive/common/metrics/TestMetrics.java e85d3f8 
>   common/src/test/org/apache/hadoop/hive/common/metrics/metrics2/TestCodahaleMetrics.java PRE-CREATION 
>   itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetaStoreMetrics.java PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java d81c856 
>   pom.xml b21d894 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 58e8e49 
>   shims/0.20S/src/main/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 6d8166c 
>   shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java 19324b8 
>   shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 5a6bc44 
> 
> Diff: https://reviews.apache.org/r/34447/diff/
> 
> 
> Testing
> -------
> 
> New unit test added.  Manually tested.
> 
> 
> Thanks,
> 
> Szehon Ho
> 
>