You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Sid Wagle <sw...@hortonworks.com> on 2016/02/02 18:58:43 UTC
Review Request 43102: AMS aggregates Counter values as average over
the timeseries (and other issues)
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43102/
-----------------------------------------------------------
Review request for Ambari, Aravindan Vijayan and Dmytro Sen.
Bugs: AMBARI-14882
https://issues.apache.org/jira/browse/AMBARI-14882
Repository: ambari
Description
-------
Current behavior:
We store the difference between two points if the metric is a counter without dividing by the time difference (Assumes fixed sampling frequency, 10 seconds for Hadoop daemons). This is similar to the Ganglia sink implementation.
Expected behavior:
- Store raw data with type = COUNTER and provide aggregate function called ._rate to calculate rate on demand.
- We need to make sure type attribute on the METRIC_RECORD data capture COUNTER / GAUGE. The HadoopMetricsSink needs to change to send this value.
Diffs
-----
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java 6d7c55f
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java e4dc423
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetricMetadata.java 0624f9c
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java 4e9e36e
ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java 0257ada
ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java f23dc42
ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java a69b7c7
ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java ff2db1d
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java 73e3de8
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java c30a354
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 53eb9d2
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/Function.java 8292657
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java c5e60fe
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java 1c1a1dc
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/FunctionTest.java af9c6bb
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStoreTest.java 8233b3f
Diff: https://reviews.apache.org/r/43102/diff/
Testing
-------
Unit test pass.
Manual testing in progress.
Thanks,
Sid Wagle
Re: Review Request 43102: AMS aggregates Counter values as average
over the timeseries (and other issues)
Posted by Aravindan Vijayan <av...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43102/#review117531
-----------------------------------------------------------
Ship it!
Ship It!
- Aravindan Vijayan
On Feb. 2, 2016, 10:39 p.m., Sid Wagle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43102/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2016, 10:39 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan and Dmytro Sen.
>
>
> Bugs: AMBARI-14882
> https://issues.apache.org/jira/browse/AMBARI-14882
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Current behavior:
> We store the difference between two points if the metric is a counter without dividing by the time difference (Assumes fixed sampling frequency, 10 seconds for Hadoop daemons). This is similar to the Ganglia sink implementation.
>
> Expected behavior:
> - Store raw data with type = COUNTER and provide aggregate function called ._rate to calculate rate on demand.
> - We need to make sure type attribute on the METRIC_RECORD data capture COUNTER / GAUGE. The HadoopMetricsSink needs to change to send this value.
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java 6d7c55f
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java e4dc423
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetricMetadata.java 0624f9c
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java 4e9e36e
> ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java 0257ada
> ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java f23dc42
> ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java a69b7c7
> ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java ff2db1d
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java 73e3de8
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java c30a354
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 53eb9d2
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/Function.java 8292657
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java c5e60fe
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java 1c1a1dc
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/FunctionTest.java af9c6bb
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStoreTest.java 8233b3f
>
> Diff: https://reviews.apache.org/r/43102/diff/
>
>
> Testing
> -------
>
> Unit test pass.
> Manual testing done.
>
>
> Thanks,
>
> Sid Wagle
>
>
Re: Review Request 43102: AMS aggregates Counter values as average
over the timeseries (and other issues)
Posted by Sid Wagle <sw...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43102/
-----------------------------------------------------------
(Updated Feb. 2, 2016, 10:39 p.m.)
Review request for Ambari, Aravindan Vijayan and Dmytro Sen.
Bugs: AMBARI-14882
https://issues.apache.org/jira/browse/AMBARI-14882
Repository: ambari
Description
-------
Current behavior:
We store the difference between two points if the metric is a counter without dividing by the time difference (Assumes fixed sampling frequency, 10 seconds for Hadoop daemons). This is similar to the Ganglia sink implementation.
Expected behavior:
- Store raw data with type = COUNTER and provide aggregate function called ._rate to calculate rate on demand.
- We need to make sure type attribute on the METRIC_RECORD data capture COUNTER / GAUGE. The HadoopMetricsSink needs to change to send this value.
Diffs
-----
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java 6d7c55f
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java e4dc423
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetricMetadata.java 0624f9c
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java 4e9e36e
ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java 0257ada
ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java f23dc42
ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java a69b7c7
ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java ff2db1d
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java 73e3de8
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java c30a354
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 53eb9d2
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/Function.java 8292657
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java c5e60fe
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java 1c1a1dc
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/FunctionTest.java af9c6bb
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStoreTest.java 8233b3f
Diff: https://reviews.apache.org/r/43102/diff/
Testing (updated)
-------
Unit test pass.
Manual testing done.
Thanks,
Sid Wagle