You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Dmytro Sen <ds...@hortonworks.com> on 2017/05/25 11:24:17 UTC
Review Request 59574: Add AMS HA support to local metrics aggregator
application
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59574/
-----------------------------------------------------------
Review request for Ambari, Aravindan Vijayan and Sid Wagle.
Bugs: AMBARI-21128
https://issues.apache.org/jira/browse/AMBARI-21128
Repository: ambari
Description
-------
Add AMS HA support for local in-memory metrics aggregation application implemented in AMBARI-20758
This include switching to working collector
Diffs
-----
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java a8dc571
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AbstractMetricPublisher.java PRE-CREATION
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AbstractMetricPublisherThread.java b1f60fa
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatedMetricsPublisher.java 0540ec9
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorApplication.java c6b703b
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/RawMetricsPublisher.java f317ed9
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/aggregator.py 2249e53
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/controller.py e5da9ba
Diff: https://reviews.apache.org/r/59574/diff/1/
Testing
-------
Unit tests + manual testing with AMS HA
Thanks,
Dmytro Sen
Re: Review Request 59574: Add AMS HA support to local metrics
aggregator application
Posted by Aravindan Vijayan <av...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59574/#review177335
-----------------------------------------------------------
Ship it!
Ship It!
- Aravindan Vijayan
On May 26, 2017, 1:03 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59574/
> -----------------------------------------------------------
>
> (Updated May 26, 2017, 1:03 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
>
>
> Bugs: AMBARI-21128
> https://issues.apache.org/jira/browse/AMBARI-21128
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Add AMS HA support for local in-memory metrics aggregation application implemented in AMBARI-20758
> This include switching to working collector
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java a8dc571
> ambari-metrics/ambari-metrics-host-aggregator/pom.xml 0598bef
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AbstractMetricPublisherThread.java b1f60fa
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatedMetricsPublisher.java 0540ec9
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorApplication.java c6b703b
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorWebService.java f96d0ed
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/RawMetricsPublisher.java f317ed9
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolder.java b355c97
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AggregatedMetricsPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/RawMetricsPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolderTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/AggregatedMetricsPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/RawMetricsPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/aggregator.py 2249e53
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/controller.py e5da9ba
>
>
> Diff: https://reviews.apache.org/r/59574/diff/2/
>
>
> Testing
> -------
>
> Unit tests + manual testing with AMS HA
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 59574: Add AMS HA support to local metrics
aggregator application
Posted by Dmytro Sen <ds...@hortonworks.com>.
> On Май 30, 2017, 7:47 п.п., Aravindan Vijayan wrote:
> > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/aggregator.py
> > Line 46 (original), 47 (patched)
> > <https://reviews.apache.org/r/59574/diff/2/?file=1733292#file1733292line47>
> >
> > FYI, This means there is no Zk fallback for getting collector hosts in the AggregatorApplication. (Like metric monitors).
The passed argument is an own hostname, not collector's. The hostname is used for sharding. Publisher will try to get working collector host from ZK if the current collector host is not available
- Dmytro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59574/#review176339
-----------------------------------------------------------
On Май 26, 2017, 1:03 п.п., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59574/
> -----------------------------------------------------------
>
> (Updated Май 26, 2017, 1:03 п.п.)
>
>
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
>
>
> Bugs: AMBARI-21128
> https://issues.apache.org/jira/browse/AMBARI-21128
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Add AMS HA support for local in-memory metrics aggregation application implemented in AMBARI-20758
> This include switching to working collector
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java a8dc571
> ambari-metrics/ambari-metrics-host-aggregator/pom.xml 0598bef
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AbstractMetricPublisherThread.java b1f60fa
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatedMetricsPublisher.java 0540ec9
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorApplication.java c6b703b
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorWebService.java f96d0ed
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/RawMetricsPublisher.java f317ed9
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolder.java b355c97
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AggregatedMetricsPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/RawMetricsPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolderTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/AggregatedMetricsPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/RawMetricsPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/aggregator.py 2249e53
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/controller.py e5da9ba
>
>
> Diff: https://reviews.apache.org/r/59574/diff/2/
>
>
> Testing
> -------
>
> Unit tests + manual testing with AMS HA
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 59574: Add AMS HA support to local metrics
aggregator application
Posted by Dmytro Sen <ds...@hortonworks.com>.
> On Май 30, 2017, 7:47 п.п., Aravindan Vijayan wrote:
> > ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisher.java
> > Lines 61 (patched)
> > <https://reviews.apache.org/r/59574/diff/2/?file=1733285#file1733285line61>
> >
> > Are we making changes such that ams-site in all hosts? If not, then this is a problem.
We do this change in all hosts. This change is already committed into ams.py as part of AMBARI-20758
- Dmytro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59574/#review176339
-----------------------------------------------------------
On Май 26, 2017, 1:03 п.п., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59574/
> -----------------------------------------------------------
>
> (Updated Май 26, 2017, 1:03 п.п.)
>
>
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
>
>
> Bugs: AMBARI-21128
> https://issues.apache.org/jira/browse/AMBARI-21128
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Add AMS HA support for local in-memory metrics aggregation application implemented in AMBARI-20758
> This include switching to working collector
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java a8dc571
> ambari-metrics/ambari-metrics-host-aggregator/pom.xml 0598bef
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AbstractMetricPublisherThread.java b1f60fa
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatedMetricsPublisher.java 0540ec9
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorApplication.java c6b703b
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorWebService.java f96d0ed
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/RawMetricsPublisher.java f317ed9
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolder.java b355c97
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AggregatedMetricsPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/RawMetricsPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolderTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/AggregatedMetricsPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/RawMetricsPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/aggregator.py 2249e53
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/controller.py e5da9ba
>
>
> Diff: https://reviews.apache.org/r/59574/diff/2/
>
>
> Testing
> -------
>
> Unit tests + manual testing with AMS HA
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 59574: Add AMS HA support to local metrics
aggregator application
Posted by Aravindan Vijayan <av...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59574/#review176339
-----------------------------------------------------------
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisher.java
Lines 61 (patched)
<https://reviews.apache.org/r/59574/#comment249738>
Are we making changes such that ams-site in all hosts? If not, then this is a problem.
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/aggregator.py
Line 46 (original), 47 (patched)
<https://reviews.apache.org/r/59574/#comment249739>
FYI, This means there is no Zk fallback for getting collector hosts in the AggregatorApplication. (Like metric monitors).
- Aravindan Vijayan
On May 26, 2017, 1:03 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59574/
> -----------------------------------------------------------
>
> (Updated May 26, 2017, 1:03 p.m.)
>
>
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
>
>
> Bugs: AMBARI-21128
> https://issues.apache.org/jira/browse/AMBARI-21128
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Add AMS HA support for local in-memory metrics aggregation application implemented in AMBARI-20758
> This include switching to working collector
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java a8dc571
> ambari-metrics/ambari-metrics-host-aggregator/pom.xml 0598bef
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AbstractMetricPublisherThread.java b1f60fa
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatedMetricsPublisher.java 0540ec9
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorApplication.java c6b703b
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorWebService.java f96d0ed
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/RawMetricsPublisher.java f317ed9
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolder.java b355c97
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AggregatedMetricsPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/RawMetricsPublisher.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolderTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/AggregatedMetricsPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/RawMetricsPublisherTest.java PRE-CREATION
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/aggregator.py 2249e53
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/controller.py e5da9ba
>
>
> Diff: https://reviews.apache.org/r/59574/diff/2/
>
>
> Testing
> -------
>
> Unit tests + manual testing with AMS HA
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 59574: Add AMS HA support to local metrics
aggregator application
Posted by Dmytro Sen <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59574/
-----------------------------------------------------------
(Updated Май 26, 2017, 1:03 п.п.)
Review request for Ambari, Aravindan Vijayan and Sid Wagle.
Bugs: AMBARI-21128
https://issues.apache.org/jira/browse/AMBARI-21128
Repository: ambari
Description
-------
Add AMS HA support for local in-memory metrics aggregation application implemented in AMBARI-20758
This include switching to working collector
Diffs (updated)
-----
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java a8dc571
ambari-metrics/ambari-metrics-host-aggregator/pom.xml 0598bef
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AbstractMetricPublisherThread.java b1f60fa
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatedMetricsPublisher.java 0540ec9
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorApplication.java c6b703b
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/AggregatorWebService.java f96d0ed
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/RawMetricsPublisher.java f317ed9
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolder.java b355c97
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisher.java PRE-CREATION
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AggregatedMetricsPublisher.java PRE-CREATION
ambari-metrics/ambari-metrics-host-aggregator/src/main/java/org/apache/hadoop/metrics2/sink/timeline/RawMetricsPublisher.java PRE-CREATION
ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/host/aggregator/TimelineMetricsHolderTest.java PRE-CREATION
ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/AbstractMetricPublisherTest.java PRE-CREATION
ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/AggregatedMetricsPublisherTest.java PRE-CREATION
ambari-metrics/ambari-metrics-host-aggregator/src/test/java/org/apache/hadoop/metrics2/sink/timeline/RawMetricsPublisherTest.java PRE-CREATION
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/aggregator.py 2249e53
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/controller.py e5da9ba
Diff: https://reviews.apache.org/r/59574/diff/2/
Changes: https://reviews.apache.org/r/59574/diff/1-2/
Testing
-------
Unit tests + manual testing with AMS HA
Thanks,
Dmytro Sen