You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2014/10/24 20:10:31 UTC
Review Request 27149: Alerts: Expose the Ability to Create Host-Only
Alerts
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27149/
-----------------------------------------------------------
Review request for Ambari, Nate Cole and Tom Beerbower.
Bugs: AMBARI-7955
https://issues.apache.org/jira/browse/AMBARI-7955
Repository: ambari
Description
-------
Adds "host-only" alerts to the alert framework infrastructure. Alerts that are only for a host are those that are run on every host in the cluster and are not bound to any particular stack service. They exist "outside" of the stack.
In order to accomplish this, several changes were made:
1) A new alerts.json was created at the root resources directory to represent host-only alerts. This file is parsed separately from the stack but its definitions are merged into the database in the same manner as those from the stack
2) These new alerts must be bound to the service AMBARI and component AMBARI_AGENT
3) Host alerts are usually custom scripts; a new directory was needed so that resources could be copied from the Ambari Server to the Agent in the same manner that the stack and custom_actions resources are copied. This is the new "host_scripts" directory.
4) RPM updates to ensure that "host_scripts" is properly packaged.
Diffs
-----
ambari-agent/pom.xml a005783
ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py a8de9f6
ambari-agent/src/main/python/ambari_agent/Controller.py 1f9471d
ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 94aa87e
ambari-agent/src/main/python/ambari_agent/FileCache.py 91ec76a
ambari-agent/src/main/python/ambari_agent/alerts/script_alert.py a00b699
ambari-agent/src/test/python/ambari_agent/TestAlerts.py ef0caae
ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 6a61c0d
ambari-server/pom.xml c66358e
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 03cf208
ambari-server/src/main/java/org/apache/ambari/server/metadata/AgentAlertDefinitions.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java 5c9bd0a
ambari-server/src/main/python/ambari-server.py d066162
ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py 1e1bdb6
ambari-server/src/main/resources/alerts.json PRE-CREATION
ambari-server/src/main/resources/host_scripts/alert_disk_space.py PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/metadata/AgentAlertDefinitionsTest.java PRE-CREATION
ambari-server/src/test/python/TestResourceFilesKeeper.py afa57fd
Diff: https://reviews.apache.org/r/27149/diff/
Testing
-------
1) Installed a developement environment on Linux
2) Apply my changes via a Git patch to the dev env and built RPMs from scratch
3) Applied RPMs to a clean VM and installed server and agent components
4) Verified that "host_scripts" is created with proper permissions and has the right contents
5) Verified that the contents of "host_scripts" are updated in accoradance with the hash update algorithm for stacks and custom_actions
6) New tests written, existing tests modified.
Thanks,
Jonathan Hurley
Re: Review Request 27149: Alerts: Expose the Ability to Create
Host-Only Alerts
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Oct. 24, 2014, 4:16 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/metadata/AgentAlertDefinitions.java, line 48
> > <https://reviews.apache.org/r/27149/diff/1/?file=732415#file732415line48>
> >
> > Copy/Paste
Fixed. I miss JDK Logging.
> On Oct. 24, 2014, 4:16 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java, lines 98-108
> > <https://reviews.apache.org/r/27149/diff/1/?file=732416#file732416line98>
> >
> > getAlertDefinitions(Reader, String) is also handling closing and exceptions?
Nice catch; documentation says that the getAlertDefinitions(...) method will do it. Fixed.
> On Oct. 24, 2014, 4:16 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/host_scripts/alert_disk_space.py, lines 83-92
> > <https://reviews.apache.org/r/27149/diff/1/?file=732420#file732420line83>
> >
> > This looks like an interview question! People are going to complain when they see an exabyte as 1000000 TB (I kid, I kid :) )
Yeah, I needed something quick and dirty to make the alert label look pretty. This brings up a larger issue of the script alerts not having any shared logic. It's not ideal to keep copying methods like this around. Something we can discuss after 2.0, I suppose.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27149/#review58340
-----------------------------------------------------------
On Oct. 24, 2014, 2:10 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27149/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 2:10 p.m.)
>
>
> Review request for Ambari, Nate Cole and Tom Beerbower.
>
>
> Bugs: AMBARI-7955
> https://issues.apache.org/jira/browse/AMBARI-7955
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Adds "host-only" alerts to the alert framework infrastructure. Alerts that are only for a host are those that are run on every host in the cluster and are not bound to any particular stack service. They exist "outside" of the stack.
>
> In order to accomplish this, several changes were made:
> 1) A new alerts.json was created at the root resources directory to represent host-only alerts. This file is parsed separately from the stack but its definitions are merged into the database in the same manner as those from the stack
>
> 2) These new alerts must be bound to the service AMBARI and component AMBARI_AGENT
>
> 3) Host alerts are usually custom scripts; a new directory was needed so that resources could be copied from the Ambari Server to the Agent in the same manner that the stack and custom_actions resources are copied. This is the new "host_scripts" directory.
>
> 4) RPM updates to ensure that "host_scripts" is properly packaged.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml a005783
> ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py a8de9f6
> ambari-agent/src/main/python/ambari_agent/Controller.py 1f9471d
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 94aa87e
> ambari-agent/src/main/python/ambari_agent/FileCache.py 91ec76a
> ambari-agent/src/main/python/ambari_agent/alerts/script_alert.py a00b699
> ambari-agent/src/test/python/ambari_agent/TestAlerts.py ef0caae
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 6a61c0d
> ambari-server/pom.xml c66358e
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 03cf208
> ambari-server/src/main/java/org/apache/ambari/server/metadata/AgentAlertDefinitions.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java 5c9bd0a
> ambari-server/src/main/python/ambari-server.py d066162
> ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py 1e1bdb6
> ambari-server/src/main/resources/alerts.json PRE-CREATION
> ambari-server/src/main/resources/host_scripts/alert_disk_space.py PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/metadata/AgentAlertDefinitionsTest.java PRE-CREATION
> ambari-server/src/test/python/TestResourceFilesKeeper.py afa57fd
>
> Diff: https://reviews.apache.org/r/27149/diff/
>
>
> Testing
> -------
>
> 1) Installed a developement environment on Linux
> 2) Apply my changes via a Git patch to the dev env and built RPMs from scratch
> 3) Applied RPMs to a clean VM and installed server and agent components
> 4) Verified that "host_scripts" is created with proper permissions and has the right contents
> 5) Verified that the contents of "host_scripts" are updated in accoradance with the hash update algorithm for stacks and custom_actions
> 6) New tests written, existing tests modified.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 27149: Alerts: Expose the Ability to Create
Host-Only Alerts
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27149/#review58340
-----------------------------------------------------------
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/metadata/AgentAlertDefinitions.java
<https://reviews.apache.org/r/27149/#comment99295>
Copy/Paste
ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java
<https://reviews.apache.org/r/27149/#comment99296>
getAlertDefinitions(Reader, String) is also handling closing and exceptions?
ambari-server/src/main/resources/host_scripts/alert_disk_space.py
<https://reviews.apache.org/r/27149/#comment99300>
This looks like an interview question! People are going to complain when they see an exabyte as 1000000 TB (I kid, I kid :) )
- Nate Cole
On Oct. 24, 2014, 2:10 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27149/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 2:10 p.m.)
>
>
> Review request for Ambari, Nate Cole and Tom Beerbower.
>
>
> Bugs: AMBARI-7955
> https://issues.apache.org/jira/browse/AMBARI-7955
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Adds "host-only" alerts to the alert framework infrastructure. Alerts that are only for a host are those that are run on every host in the cluster and are not bound to any particular stack service. They exist "outside" of the stack.
>
> In order to accomplish this, several changes were made:
> 1) A new alerts.json was created at the root resources directory to represent host-only alerts. This file is parsed separately from the stack but its definitions are merged into the database in the same manner as those from the stack
>
> 2) These new alerts must be bound to the service AMBARI and component AMBARI_AGENT
>
> 3) Host alerts are usually custom scripts; a new directory was needed so that resources could be copied from the Ambari Server to the Agent in the same manner that the stack and custom_actions resources are copied. This is the new "host_scripts" directory.
>
> 4) RPM updates to ensure that "host_scripts" is properly packaged.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml a005783
> ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py a8de9f6
> ambari-agent/src/main/python/ambari_agent/Controller.py 1f9471d
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 94aa87e
> ambari-agent/src/main/python/ambari_agent/FileCache.py 91ec76a
> ambari-agent/src/main/python/ambari_agent/alerts/script_alert.py a00b699
> ambari-agent/src/test/python/ambari_agent/TestAlerts.py ef0caae
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 6a61c0d
> ambari-server/pom.xml c66358e
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 03cf208
> ambari-server/src/main/java/org/apache/ambari/server/metadata/AgentAlertDefinitions.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java 5c9bd0a
> ambari-server/src/main/python/ambari-server.py d066162
> ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py 1e1bdb6
> ambari-server/src/main/resources/alerts.json PRE-CREATION
> ambari-server/src/main/resources/host_scripts/alert_disk_space.py PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/metadata/AgentAlertDefinitionsTest.java PRE-CREATION
> ambari-server/src/test/python/TestResourceFilesKeeper.py afa57fd
>
> Diff: https://reviews.apache.org/r/27149/diff/
>
>
> Testing
> -------
>
> 1) Installed a developement environment on Linux
> 2) Apply my changes via a Git patch to the dev env and built RPMs from scratch
> 3) Applied RPMs to a clean VM and installed server and agent components
> 4) Verified that "host_scripts" is created with proper permissions and has the right contents
> 5) Verified that the contents of "host_scripts" are updated in accoradance with the hash update algorithm for stacks and custom_actions
> 6) New tests written, existing tests modified.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 27149: Alerts: Expose the Ability to Create
Host-Only Alerts
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Oct. 24, 2014, 3:49 p.m., Tom Beerbower wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java, line 1331
> > <https://reviews.apache.org/r/27149/diff/1/?file=732414#file732414line1331>
> >
> > is the continue needed here?
No, it's not. Leftover from when we were still merging updates from the file system into the DB. Fixed.
> On Oct. 24, 2014, 3:49 p.m., Tom Beerbower wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java, lines 102-103
> > <https://reviews.apache.org/r/27149/diff/1/?file=732416#file732416line102>
> >
> > nit : maybe instead of duplicating the error text, extract a single string to reuse.
Extracted.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27149/#review58333
-----------------------------------------------------------
On Oct. 24, 2014, 2:10 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27149/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 2:10 p.m.)
>
>
> Review request for Ambari, Nate Cole and Tom Beerbower.
>
>
> Bugs: AMBARI-7955
> https://issues.apache.org/jira/browse/AMBARI-7955
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Adds "host-only" alerts to the alert framework infrastructure. Alerts that are only for a host are those that are run on every host in the cluster and are not bound to any particular stack service. They exist "outside" of the stack.
>
> In order to accomplish this, several changes were made:
> 1) A new alerts.json was created at the root resources directory to represent host-only alerts. This file is parsed separately from the stack but its definitions are merged into the database in the same manner as those from the stack
>
> 2) These new alerts must be bound to the service AMBARI and component AMBARI_AGENT
>
> 3) Host alerts are usually custom scripts; a new directory was needed so that resources could be copied from the Ambari Server to the Agent in the same manner that the stack and custom_actions resources are copied. This is the new "host_scripts" directory.
>
> 4) RPM updates to ensure that "host_scripts" is properly packaged.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml a005783
> ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py a8de9f6
> ambari-agent/src/main/python/ambari_agent/Controller.py 1f9471d
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 94aa87e
> ambari-agent/src/main/python/ambari_agent/FileCache.py 91ec76a
> ambari-agent/src/main/python/ambari_agent/alerts/script_alert.py a00b699
> ambari-agent/src/test/python/ambari_agent/TestAlerts.py ef0caae
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 6a61c0d
> ambari-server/pom.xml c66358e
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 03cf208
> ambari-server/src/main/java/org/apache/ambari/server/metadata/AgentAlertDefinitions.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java 5c9bd0a
> ambari-server/src/main/python/ambari-server.py d066162
> ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py 1e1bdb6
> ambari-server/src/main/resources/alerts.json PRE-CREATION
> ambari-server/src/main/resources/host_scripts/alert_disk_space.py PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/metadata/AgentAlertDefinitionsTest.java PRE-CREATION
> ambari-server/src/test/python/TestResourceFilesKeeper.py afa57fd
>
> Diff: https://reviews.apache.org/r/27149/diff/
>
>
> Testing
> -------
>
> 1) Installed a developement environment on Linux
> 2) Apply my changes via a Git patch to the dev env and built RPMs from scratch
> 3) Applied RPMs to a clean VM and installed server and agent components
> 4) Verified that "host_scripts" is created with proper permissions and has the right contents
> 5) Verified that the contents of "host_scripts" are updated in accoradance with the hash update algorithm for stacks and custom_actions
> 6) New tests written, existing tests modified.
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 27149: Alerts: Expose the Ability to Create
Host-Only Alerts
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27149/#review58333
-----------------------------------------------------------
Ship it!
a couple minor comments.
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
<https://reviews.apache.org/r/27149/#comment99285>
is the continue needed here?
ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java
<https://reviews.apache.org/r/27149/#comment99286>
nit : maybe instead of duplicating the error text, extract a single string to reuse.
- Tom Beerbower
On Oct. 24, 2014, 6:10 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27149/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 6:10 p.m.)
>
>
> Review request for Ambari, Nate Cole and Tom Beerbower.
>
>
> Bugs: AMBARI-7955
> https://issues.apache.org/jira/browse/AMBARI-7955
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Adds "host-only" alerts to the alert framework infrastructure. Alerts that are only for a host are those that are run on every host in the cluster and are not bound to any particular stack service. They exist "outside" of the stack.
>
> In order to accomplish this, several changes were made:
> 1) A new alerts.json was created at the root resources directory to represent host-only alerts. This file is parsed separately from the stack but its definitions are merged into the database in the same manner as those from the stack
>
> 2) These new alerts must be bound to the service AMBARI and component AMBARI_AGENT
>
> 3) Host alerts are usually custom scripts; a new directory was needed so that resources could be copied from the Ambari Server to the Agent in the same manner that the stack and custom_actions resources are copied. This is the new "host_scripts" directory.
>
> 4) RPM updates to ensure that "host_scripts" is properly packaged.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml a005783
> ambari-agent/src/main/python/ambari_agent/AlertSchedulerHandler.py a8de9f6
> ambari-agent/src/main/python/ambari_agent/Controller.py 1f9471d
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 94aa87e
> ambari-agent/src/main/python/ambari_agent/FileCache.py 91ec76a
> ambari-agent/src/main/python/ambari_agent/alerts/script_alert.py a00b699
> ambari-agent/src/test/python/ambari_agent/TestAlerts.py ef0caae
> ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py 6a61c0d
> ambari-server/pom.xml c66358e
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 03cf208
> ambari-server/src/main/java/org/apache/ambari/server/metadata/AgentAlertDefinitions.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/AlertDefinitionFactory.java 5c9bd0a
> ambari-server/src/main/python/ambari-server.py d066162
> ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py 1e1bdb6
> ambari-server/src/main/resources/alerts.json PRE-CREATION
> ambari-server/src/main/resources/host_scripts/alert_disk_space.py PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/metadata/AgentAlertDefinitionsTest.java PRE-CREATION
> ambari-server/src/test/python/TestResourceFilesKeeper.py afa57fd
>
> Diff: https://reviews.apache.org/r/27149/diff/
>
>
> Testing
> -------
>
> 1) Installed a developement environment on Linux
> 2) Apply my changes via a Git patch to the dev env and built RPMs from scratch
> 3) Applied RPMs to a clean VM and installed server and agent components
> 4) Verified that "host_scripts" is created with proper permissions and has the right contents
> 5) Verified that the contents of "host_scripts" are updated in accoradance with the hash update algorithm for stacks and custom_actions
> 6) New tests written, existing tests modified.
>
>
> Thanks,
>
> Jonathan Hurley
>
>