You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2016/03/04 18:46:11 UTC
Review Request 44397: New Alerts Do Not Honor Existing Maintenance
Mode Setting
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44397/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jayush Luniya, and Sumit Mohanty.
Bugs: AMBARI-15303
https://issues.apache.org/jira/browse/AMBARI-15303
Repository: ambari
Description
-------
Alerts "suppress" maintenance mode by indicating a {{maintenance_state}} attribute in addition to the actual state which is being reported:
{code}
"Alert": {
"cluster_name": "c1",
"component_name": "METRICS_COLLECTOR",
"definition_id": 43,
"definition_name": "ams_metrics_collector_process",
"host_name": "c6401.ambari.apache.org",
"id": 28,
"instance": null,
"label": "Metrics Collector Process",
"latest_timestamp": 1457108946118,
"maintenance_state": "ON",
"original_timestamp": 1457108646099,
"scope": "ANY",
"service_name": "AMBARI_METRICS",
"state": "CRITICAL",
"text": "Connection failed: [Errno 111] Connection refused to c6401.ambari.apache.org"
}
{code}
When a host/service/component is placed into MM, the database is updated so that all {{alert_current}} rows which are affected have their MM updated as well.
However, this fails under two scenarios:
- The alert hasn't been received yet in a brand new cluster
- The alert definition was disabled, which removed all current alerts. Then, it was re-enabled.
In both cases, when constructing a new {{AlertCurrentEntity}}, we need to calculate the correct maintenance state.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java cd49e76
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java 9bbfe37
Diff: https://reviews.apache.org/r/44397/diff/
Testing
-------
PENDING: Writing UTs and running tests now...
Verified fix in an existing cluster by disabling alerts, then re-enabling them on a MM component with an active alert.
Thanks,
Jonathan Hurley
Re: Review Request 44397: New Alerts Do Not Honor Existing
Maintenance Mode Setting
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44397/#review122113
-----------------------------------------------------------
Ship it!
Ship It!
- Sumit Mohanty
On March 4, 2016, 5:46 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44397/
> -----------------------------------------------------------
>
> (Updated March 4, 2016, 5:46 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jayush Luniya, and Sumit Mohanty.
>
>
> Bugs: AMBARI-15303
> https://issues.apache.org/jira/browse/AMBARI-15303
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Alerts "suppress" maintenance mode by indicating a {{maintenance_state}} attribute in addition to the actual state which is being reported:
>
> {code}
> "Alert": {
> "cluster_name": "c1",
> "component_name": "METRICS_COLLECTOR",
> "definition_id": 43,
> "definition_name": "ams_metrics_collector_process",
> "host_name": "c6401.ambari.apache.org",
> "id": 28,
> "instance": null,
> "label": "Metrics Collector Process",
> "latest_timestamp": 1457108946118,
> "maintenance_state": "ON",
> "original_timestamp": 1457108646099,
> "scope": "ANY",
> "service_name": "AMBARI_METRICS",
> "state": "CRITICAL",
> "text": "Connection failed: [Errno 111] Connection refused to c6401.ambari.apache.org"
> }
> {code}
>
> When a host/service/component is placed into MM, the database is updated so that all {{alert_current}} rows which are affected have their MM updated as well.
>
> However, this fails under two scenarios:
> - The alert hasn't been received yet in a brand new cluster
> - The alert definition was disabled, which removed all current alerts. Then, it was re-enabled.
>
> In both cases, when constructing a new {{AlertCurrentEntity}}, we need to calculate the correct maintenance state.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java cd49e76
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java 9bbfe37
>
> Diff: https://reviews.apache.org/r/44397/diff/
>
>
> Testing
> -------
>
> PENDING: Writing UTs and running tests now...
>
> Verified fix in an existing cluster by disabling alerts, then re-enabling them on a MM component with an active alert.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 44397: New Alerts Do Not Honor Existing
Maintenance Mode Setting
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On March 4, 2016, 12:51 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java, line 248
> > <https://reviews.apache.org/r/44397/diff/1/?file=1281067#file1281067line248>
> >
> > We did not have a helper method yet for this? This is useful - I will let Nahappan know. I think he needed something like this for something he is working on.
We did not; never needed it before now since it was all handled in the AlertMaintenanceModeListener which didn't need to calculate the state since it already had it from the fired event.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44397/#review122112
-----------------------------------------------------------
On March 4, 2016, 12:46 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44397/
> -----------------------------------------------------------
>
> (Updated March 4, 2016, 12:46 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jayush Luniya, and Sumit Mohanty.
>
>
> Bugs: AMBARI-15303
> https://issues.apache.org/jira/browse/AMBARI-15303
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Alerts "suppress" maintenance mode by indicating a {{maintenance_state}} attribute in addition to the actual state which is being reported:
>
> {code}
> "Alert": {
> "cluster_name": "c1",
> "component_name": "METRICS_COLLECTOR",
> "definition_id": 43,
> "definition_name": "ams_metrics_collector_process",
> "host_name": "c6401.ambari.apache.org",
> "id": 28,
> "instance": null,
> "label": "Metrics Collector Process",
> "latest_timestamp": 1457108946118,
> "maintenance_state": "ON",
> "original_timestamp": 1457108646099,
> "scope": "ANY",
> "service_name": "AMBARI_METRICS",
> "state": "CRITICAL",
> "text": "Connection failed: [Errno 111] Connection refused to c6401.ambari.apache.org"
> }
> {code}
>
> When a host/service/component is placed into MM, the database is updated so that all {{alert_current}} rows which are affected have their MM updated as well.
>
> However, this fails under two scenarios:
> - The alert hasn't been received yet in a brand new cluster
> - The alert definition was disabled, which removed all current alerts. Then, it was re-enabled.
>
> In both cases, when constructing a new {{AlertCurrentEntity}}, we need to calculate the correct maintenance state.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java cd49e76
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java 9bbfe37
>
> Diff: https://reviews.apache.org/r/44397/diff/
>
>
> Testing
> -------
>
> PENDING: Writing UTs and running tests now...
>
> Verified fix in an existing cluster by disabling alerts, then re-enabling them on a MM component with an active alert.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 44397: New Alerts Do Not Honor Existing
Maintenance Mode Setting
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44397/#review122112
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java (line 248)
<https://reviews.apache.org/r/44397/#comment184021>
We did not have a helper method yet for this? This is useful - I will let Nahappan know. I think he needed something like this for something he is working on.
- Sumit Mohanty
On March 4, 2016, 5:46 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44397/
> -----------------------------------------------------------
>
> (Updated March 4, 2016, 5:46 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jayush Luniya, and Sumit Mohanty.
>
>
> Bugs: AMBARI-15303
> https://issues.apache.org/jira/browse/AMBARI-15303
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Alerts "suppress" maintenance mode by indicating a {{maintenance_state}} attribute in addition to the actual state which is being reported:
>
> {code}
> "Alert": {
> "cluster_name": "c1",
> "component_name": "METRICS_COLLECTOR",
> "definition_id": 43,
> "definition_name": "ams_metrics_collector_process",
> "host_name": "c6401.ambari.apache.org",
> "id": 28,
> "instance": null,
> "label": "Metrics Collector Process",
> "latest_timestamp": 1457108946118,
> "maintenance_state": "ON",
> "original_timestamp": 1457108646099,
> "scope": "ANY",
> "service_name": "AMBARI_METRICS",
> "state": "CRITICAL",
> "text": "Connection failed: [Errno 111] Connection refused to c6401.ambari.apache.org"
> }
> {code}
>
> When a host/service/component is placed into MM, the database is updated so that all {{alert_current}} rows which are affected have their MM updated as well.
>
> However, this fails under two scenarios:
> - The alert hasn't been received yet in a brand new cluster
> - The alert definition was disabled, which removed all current alerts. Then, it was re-enabled.
>
> In both cases, when constructing a new {{AlertCurrentEntity}}, we need to calculate the correct maintenance state.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java cd49e76
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java 9bbfe37
>
> Diff: https://reviews.apache.org/r/44397/diff/
>
>
> Testing
> -------
>
> PENDING: Writing UTs and running tests now...
>
> Verified fix in an existing cluster by disabling alerts, then re-enabling them on a MM component with an active alert.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 44397: New Alerts Do Not Honor Existing
Maintenance Mode Setting
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44397/#review122136
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On March 4, 2016, 6:27 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44397/
> -----------------------------------------------------------
>
> (Updated March 4, 2016, 6:27 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jayush Luniya, and Sumit Mohanty.
>
>
> Bugs: AMBARI-15303
> https://issues.apache.org/jira/browse/AMBARI-15303
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Alerts "suppress" maintenance mode by indicating a {{maintenance_state}} attribute in addition to the actual state which is being reported:
>
> {code}
> "Alert": {
> "cluster_name": "c1",
> "component_name": "METRICS_COLLECTOR",
> "definition_id": 43,
> "definition_name": "ams_metrics_collector_process",
> "host_name": "c6401.ambari.apache.org",
> "id": 28,
> "instance": null,
> "label": "Metrics Collector Process",
> "latest_timestamp": 1457108946118,
> "maintenance_state": "ON",
> "original_timestamp": 1457108646099,
> "scope": "ANY",
> "service_name": "AMBARI_METRICS",
> "state": "CRITICAL",
> "text": "Connection failed: [Errno 111] Connection refused to c6401.ambari.apache.org"
> }
> {code}
>
> When a host/service/component is placed into MM, the database is updated so that all {{alert_current}} rows which are affected have their MM updated as well.
>
> However, this fails under two scenarios:
> - The alert hasn't been received yet in a brand new cluster
> - The alert definition was disabled, which removed all current alerts. Then, it was re-enabled.
>
> In both cases, when constructing a new {{AlertCurrentEntity}}, we need to calculate the correct maintenance state.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java cd49e76
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java 9bbfe37
> ambari-server/src/test/java/org/apache/ambari/server/controller/MaintenanceStateHelperTest.java d9c5039
> ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertReceivedListenerTest.java 6e58876
>
> Diff: https://reviews.apache.org/r/44397/diff/
>
>
> Testing
> -------
>
> PENDING: Writing UTs and running tests now...
>
> Verified fix in an existing cluster by disabling alerts, then re-enabling them on a MM component with an active alert.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 44397: New Alerts Do Not Honor Existing
Maintenance Mode Setting
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44397/
-----------------------------------------------------------
(Updated March 4, 2016, 2:25 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jayush Luniya, and Sumit Mohanty.
Bugs: AMBARI-15303
https://issues.apache.org/jira/browse/AMBARI-15303
Repository: ambari
Description
-------
Alerts "suppress" maintenance mode by indicating a {{maintenance_state}} attribute in addition to the actual state which is being reported:
{code}
"Alert": {
"cluster_name": "c1",
"component_name": "METRICS_COLLECTOR",
"definition_id": 43,
"definition_name": "ams_metrics_collector_process",
"host_name": "c6401.ambari.apache.org",
"id": 28,
"instance": null,
"label": "Metrics Collector Process",
"latest_timestamp": 1457108946118,
"maintenance_state": "ON",
"original_timestamp": 1457108646099,
"scope": "ANY",
"service_name": "AMBARI_METRICS",
"state": "CRITICAL",
"text": "Connection failed: [Errno 111] Connection refused to c6401.ambari.apache.org"
}
{code}
When a host/service/component is placed into MM, the database is updated so that all {{alert_current}} rows which are affected have their MM updated as well.
However, this fails under two scenarios:
- The alert hasn't been received yet in a brand new cluster
- The alert definition was disabled, which removed all current alerts. Then, it was re-enabled.
In both cases, when constructing a new {{AlertCurrentEntity}}, we need to calculate the correct maintenance state.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java cd49e76
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java 9bbfe37
ambari-server/src/test/java/org/apache/ambari/server/controller/MaintenanceStateHelperTest.java d9c5039
ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertReceivedListenerTest.java 6e58876
Diff: https://reviews.apache.org/r/44397/diff/
Testing (updated)
-------
Tests run: 3918, Failures: 0, Errors: 0, Skipped: 33
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:06 min
[INFO] Finished at: 2016-03-04T13:32:30-05:00
[INFO] Final Memory: 50M/607M
[INFO] ------------------------------------------------------------------------
Verified fix in an existing cluster by disabling alerts, then re-enabling them on a MM component with an active alert.
Thanks,
Jonathan Hurley
Re: Review Request 44397: New Alerts Do Not Honor Existing
Maintenance Mode Setting
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44397/
-----------------------------------------------------------
(Updated March 4, 2016, 1:27 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jayush Luniya, and Sumit Mohanty.
Changes
-------
Updated with tests.
Bugs: AMBARI-15303
https://issues.apache.org/jira/browse/AMBARI-15303
Repository: ambari
Description
-------
Alerts "suppress" maintenance mode by indicating a {{maintenance_state}} attribute in addition to the actual state which is being reported:
{code}
"Alert": {
"cluster_name": "c1",
"component_name": "METRICS_COLLECTOR",
"definition_id": 43,
"definition_name": "ams_metrics_collector_process",
"host_name": "c6401.ambari.apache.org",
"id": 28,
"instance": null,
"label": "Metrics Collector Process",
"latest_timestamp": 1457108946118,
"maintenance_state": "ON",
"original_timestamp": 1457108646099,
"scope": "ANY",
"service_name": "AMBARI_METRICS",
"state": "CRITICAL",
"text": "Connection failed: [Errno 111] Connection refused to c6401.ambari.apache.org"
}
{code}
When a host/service/component is placed into MM, the database is updated so that all {{alert_current}} rows which are affected have their MM updated as well.
However, this fails under two scenarios:
- The alert hasn't been received yet in a brand new cluster
- The alert definition was disabled, which removed all current alerts. Then, it was re-enabled.
In both cases, when constructing a new {{AlertCurrentEntity}}, we need to calculate the correct maintenance state.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/MaintenanceStateHelper.java cd49e76
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java 9bbfe37
ambari-server/src/test/java/org/apache/ambari/server/controller/MaintenanceStateHelperTest.java d9c5039
ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertReceivedListenerTest.java 6e58876
Diff: https://reviews.apache.org/r/44397/diff/
Testing
-------
PENDING: Writing UTs and running tests now...
Verified fix in an existing cluster by disabling alerts, then re-enabling them on a MM component with an active alert.
Thanks,
Jonathan Hurley