You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Aravindan Vijayan <av...@hortonworks.com> on 2015/11/03 18:35:00 UTC
Review Request 39907: Introduce cluster wide MINUTE aggregator in
Ambari Metrics service
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39907/
-----------------------------------------------------------
Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-13701
https://issues.apache.org/jira/browse/AMBARI-13701
Repository: ambari
Description
-------
PURPOSE
Currently, AMS has aggregators and aggregated tables only for SECONDS, HOURS and DAYS. We need to introduce the MINUTE aggregator and the corresponding table to return MINUTE granularity data for appropriate time ranges.
FIX
Added the minute aggregator and changed the current 30 second aggregator to be called as the "SECOND" aggregator.
Diffs
-----
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 17df629
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 3ce30fd
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java 30e42f2
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java f07918c
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java 092c983
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java cbf0233
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java 89fee7c
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheEntryFactory.java 7c7db9f
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java 8f1a481
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml 89b584b
Diff: https://reviews.apache.org/r/39907/diff/
Testing
-------
Relevant unit tests pass.
Manual testing done.
Added new unit test to cover Minute aggregator.
Thanks,
Aravindan Vijayan
Re: Review Request 39907: Introduce cluster wide MINUTE aggregator in
Ambari Metrics service
Posted by Sid Wagle <sw...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39907/#review104956
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java (line 837)
<https://reviews.apache.org/r/39907/#comment163312>
Wrong config type.
Unit test updates missing.
- Sid Wagle
On Nov. 3, 2015, 5:35 p.m., Aravindan Vijayan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39907/
> -----------------------------------------------------------
>
> (Updated Nov. 3, 2015, 5:35 p.m.)
>
>
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-13701
> https://issues.apache.org/jira/browse/AMBARI-13701
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PURPOSE
> Currently, AMS has aggregators and aggregated tables only for SECONDS, HOURS and DAYS. We need to introduce the MINUTE aggregator and the corresponding table to return MINUTE granularity data for appropriate time ranges.
>
> FIX
> Added the minute aggregator and changed the current 30 second aggregator to be called as the "SECOND" aggregator.
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 17df629
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 3ce30fd
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java 30e42f2
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java f07918c
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java 092c983
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java cbf0233
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java 89fee7c
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheEntryFactory.java 7c7db9f
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java 8f1a481
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml 89b584b
>
> Diff: https://reviews.apache.org/r/39907/diff/
>
>
> Testing
> -------
>
> Relevant unit tests pass.
>
> Manual testing done.
>
> Added new unit test to cover Minute aggregator.
>
>
> Thanks,
>
> Aravindan Vijayan
>
>
Re: Review Request 39907: Introduce cluster wide MINUTE aggregator in
Ambari Metrics service
Posted by Sid Wagle <sw...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39907/#review104951
-----------------------------------------------------------
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java (line 243)
<https://reviews.apache.org/r/39907/#comment163303>
This should be renamed to Seconds
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java (line 387)
<https://reviews.apache.org/r/39907/#comment163304>
formatting
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheEntryFactory.java (line 254)
<https://reviews.apache.org/r/39907/#comment163306>
formatting.
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheEntryFactory.java (line 262)
<https://reviews.apache.org/r/39907/#comment163308>
Can we get away with not creatign a TreeMap.
- Sid Wagle
On Nov. 3, 2015, 5:35 p.m., Aravindan Vijayan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39907/
> -----------------------------------------------------------
>
> (Updated Nov. 3, 2015, 5:35 p.m.)
>
>
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-13701
> https://issues.apache.org/jira/browse/AMBARI-13701
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PURPOSE
> Currently, AMS has aggregators and aggregated tables only for SECONDS, HOURS and DAYS. We need to introduce the MINUTE aggregator and the corresponding table to return MINUTE granularity data for appropriate time ranges.
>
> FIX
> Added the minute aggregator and changed the current 30 second aggregator to be called as the "SECOND" aggregator.
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 17df629
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 3ce30fd
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java 30e42f2
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java f07918c
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java 092c983
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java cbf0233
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java 89fee7c
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheEntryFactory.java 7c7db9f
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java 8f1a481
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml 89b584b
>
> Diff: https://reviews.apache.org/r/39907/diff/
>
>
> Testing
> -------
>
> Relevant unit tests pass.
>
> Manual testing done.
>
> Added new unit test to cover Minute aggregator.
>
>
> Thanks,
>
> Aravindan Vijayan
>
>
Re: Review Request 39907: Introduce cluster wide MINUTE aggregator in
Ambari Metrics service
Posted by Sid Wagle <sw...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39907/#review105037
-----------------------------------------------------------
Ship it!
Ship It!
- Sid Wagle
On Nov. 4, 2015, 5:20 a.m., Aravindan Vijayan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39907/
> -----------------------------------------------------------
>
> (Updated Nov. 4, 2015, 5:20 a.m.)
>
>
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-13701
> https://issues.apache.org/jira/browse/AMBARI-13701
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PURPOSE
> Currently, AMS has aggregators and aggregated tables only for SECONDS, HOURS and DAYS. We need to introduce the MINUTE aggregator and the corresponding table to return MINUTE granularity data for appropriate time ranges.
>
> FIX
> Added the minute aggregator and changed the current 30 second aggregator to be called as the "SECOND" aggregator.
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 17df629
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 3ce30fd
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java 30e42f2
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java f07918c
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorMinute.java 85bdbbc
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java PRE-CREATION
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java 092c983
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java cbf0233
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java 89fee7c
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheEntryFactory.java 7c7db9f
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java 8f1a481
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml 89b584b
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py cd9c844
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java eb4d677
>
> Diff: https://reviews.apache.org/r/39907/diff/
>
>
> Testing
> -------
>
> Relevant unit tests pass.
>
> Manual testing done.
>
> Added new unit test to cover Minute aggregator.
>
>
> Thanks,
>
> Aravindan Vijayan
>
>
Re: Review Request 39907: Introduce cluster wide MINUTE aggregator in
Ambari Metrics service
Posted by Aravindan Vijayan <av...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39907/
-----------------------------------------------------------
(Updated Nov. 4, 2015, 5:20 a.m.)
Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
Changes
-------
Incorporated review changes.
Bugs: AMBARI-13701
https://issues.apache.org/jira/browse/AMBARI-13701
Repository: ambari
Description
-------
PURPOSE
Currently, AMS has aggregators and aggregated tables only for SECONDS, HOURS and DAYS. We need to introduce the MINUTE aggregator and the corresponding table to return MINUTE granularity data for appropriate time ranges.
FIX
Added the minute aggregator and changed the current 30 second aggregator to be called as the "SECOND" aggregator.
Diffs (updated)
-----
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 17df629
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 3ce30fd
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java 30e42f2
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricAggregatorFactory.java f07918c
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorMinute.java 85bdbbc
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricClusterAggregatorSecond.java PRE-CREATION
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java 092c983
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java cbf0233
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java 89fee7c
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheEntryFactory.java 7c7db9f
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java 8f1a481
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml 89b584b
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py cd9c844
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java eb4d677
Diff: https://reviews.apache.org/r/39907/diff/
Testing
-------
Relevant unit tests pass.
Manual testing done.
Added new unit test to cover Minute aggregator.
Thanks,
Aravindan Vijayan