You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Vitalyi Brodetskyi <vb...@hortonworks.com> on 2016/11/29 22:52:34 UTC
Review Request 54187: Perf: Simulate alerts for multiple Ambari Agents
running on single Host
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54187/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez and Sid Wagle.
Bugs: AMBARI-18736
https://issues.apache.org/jira/browse/AMBARI-18736
Repository: ambari
Description
-------
The PERF stack can actually create Python scripts to run actual alerts whose behavior is controlled by configs to either always pass, always fail, pass with some success %, flip/thrash, timeout, etc.
For other types of alerts (web, host:port), we will need to override the Alert Scheduler on the Ambari Agent to return fake values.
The goal is to stress test Ambari Server when multiple Ambari Agents run on a single host.
Diffs
-----
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/metainfo.xml 36a2168
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/package/alerts/alert_happy_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/alerts.json 1b3ae25
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metainfo.xml b304706
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_master_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_regionserver_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/alerts.json 8ccfa47
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/metainfo.xml bf26fbd
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_checkpoint_time.py 1e69def
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_datanode_unmounted_data_dir.py 8c122ae
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_ha_namenode_health.py 7f03d88
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_metrics_deviation.py 0946c85
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_nfs_gateway_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_snamenode_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_upgrade_finalized.py 0dff200
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/configuration/sleepy-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/metainfo.xml 470d809
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/package/alerts/alert_sleepy_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/configuration/snow-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/metainfo.xml cbb57f6
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/package/alerts/alert_snow_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/alerts.json c4a58bb
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/metainfo.xml 8b18e73
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_history_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanager_health.py d2cfe26
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanagers_summary.py 81527f6
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_resourcemanager_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_timeline_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/configuration/zk-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/metainfo.xml 62ea041
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/package/alerts/alert_zk_server_process.py PRE-CREATION
Diff: https://reviews.apache.org/r/54187/diff/
Testing
-------
mvn clean test
Thanks,
Vitalyi Brodetskyi
Re: Review Request 54187: Perf: Simulate alerts for multiple Ambari
Agents running on single Host
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
> On \u041b\u0438\u0441. 30, 2016, 12:02 \u0434\u043e \u043f\u043e\u043b\u0443\u0434\u043d\u044f, Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml, line 20
> > <https://reviews.apache.org/r/54187/diff/1/?file=1572589#file1572589line20>
> >
> > Can we add these configs to the theme in the UI so we can use a dropdown for the behavior, slider for percentage from 0-100, dropdown for return value, and textbox for the secs/mins?
> > Feel free to do that in a separate Jira.
Sure, will create new jira.
- Vitalyi
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54187/#review157353
-----------------------------------------------------------
On \u041b\u0438\u0441. 30, 2016, 11:29 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f, Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54187/
> -----------------------------------------------------------
>
> (Updated \u041b\u0438\u0441. 30, 2016, 11:29 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f)
>
>
> Review request for Ambari, Alejandro Fernandez and Sid Wagle.
>
>
> Bugs: AMBARI-18736
> https://issues.apache.org/jira/browse/AMBARI-18736
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The PERF stack can actually create Python scripts to run actual alerts whose behavior is controlled by configs to either always pass, always fail, pass with some success %, flip/thrash, timeout, etc.
>
> For other types of alerts (web, host:port), we will need to override the Alert Scheduler on the Ambari Agent to return fake values.
>
> The goal is to stress test Ambari Server when multiple Ambari Agents run on a single host.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/simulate_perf_cluster_alert_behaviour.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/metainfo.xml 36a2168
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/package/alerts/alert_happy_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/alerts.json 1b3ae25
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metainfo.xml b304706
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_master_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_regionserver_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/alerts.json 8ccfa47
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/metainfo.xml bf26fbd
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_checkpoint_time.py 1e69def
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_datanode_unmounted_data_dir.py 8c122ae
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_ha_namenode_health.py 7f03d88
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_metrics_deviation.py 0946c85
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_nfs_gateway_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_snamenode_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_upgrade_finalized.py 0dff200
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/configuration/sleepy-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/metainfo.xml 470d809
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/package/alerts/alert_sleepy_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/configuration/snow-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/metainfo.xml cbb57f6
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/package/alerts/alert_snow_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/alerts.json c4a58bb
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/metainfo.xml 8b18e73
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_history_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanager_health.py d2cfe26
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanagers_summary.py 81527f6
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_resourcemanager_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_timeline_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/configuration/zk-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/metainfo.xml 62ea041
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/package/alerts/alert_zk_server_process.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/54187/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 54187: Perf: Simulate alerts for multiple Ambari
Agents running on single Host
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54187/#review157353
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml (line 20)
<https://reviews.apache.org/r/54187/#comment227913>
Can we add these configs to the theme in the UI so we can use a dropdown for the behavior, slider for percentage from 0-100, dropdown for return value, and textbox for the secs/mins?
Feel free to do that in a separate Jira.
- Alejandro Fernandez
On Nov. 29, 2016, 10:52 p.m., Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54187/
> -----------------------------------------------------------
>
> (Updated Nov. 29, 2016, 10:52 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Sid Wagle.
>
>
> Bugs: AMBARI-18736
> https://issues.apache.org/jira/browse/AMBARI-18736
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The PERF stack can actually create Python scripts to run actual alerts whose behavior is controlled by configs to either always pass, always fail, pass with some success %, flip/thrash, timeout, etc.
>
> For other types of alerts (web, host:port), we will need to override the Alert Scheduler on the Ambari Agent to return fake values.
>
> The goal is to stress test Ambari Server when multiple Ambari Agents run on a single host.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/metainfo.xml 36a2168
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/package/alerts/alert_happy_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/alerts.json 1b3ae25
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metainfo.xml b304706
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_master_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_regionserver_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/alerts.json 8ccfa47
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/metainfo.xml bf26fbd
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_checkpoint_time.py 1e69def
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_datanode_unmounted_data_dir.py 8c122ae
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_ha_namenode_health.py 7f03d88
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_metrics_deviation.py 0946c85
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_nfs_gateway_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_snamenode_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_upgrade_finalized.py 0dff200
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/configuration/sleepy-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/metainfo.xml 470d809
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/package/alerts/alert_sleepy_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/configuration/snow-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/metainfo.xml cbb57f6
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/package/alerts/alert_snow_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/alerts.json c4a58bb
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/metainfo.xml 8b18e73
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_history_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanager_health.py d2cfe26
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanagers_summary.py 81527f6
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_resourcemanager_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_timeline_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/configuration/zk-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/metainfo.xml 62ea041
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/package/alerts/alert_zk_server_process.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/54187/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 54187: Perf: Simulate alerts for multiple Ambari
Agents running on single Host
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54187/#review157350
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/package/alerts/alert_happy_process.py (line 58)
<https://reviews.apache.org/r/54187/#comment227899>
Let's make all of these python alert files inherit from a file called dummy_alert.py (somewhere in ambari-common).
Each service has slightly different configs it needs based on the config-type, but the body of the execute function should be identical for all of them.
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/package/alerts/alert_happy_process.py (line 94)
<https://reviews.apache.org/r/54187/#comment227900>
If either one is missing, raise an exception.
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/package/alerts/alert_happy_process.py (line 97)
<https://reviews.apache.org/r/54187/#comment227901>
Please add a print statement that it's sleeping.
- Alejandro Fernandez
On Nov. 29, 2016, 10:52 p.m., Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54187/
> -----------------------------------------------------------
>
> (Updated Nov. 29, 2016, 10:52 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Sid Wagle.
>
>
> Bugs: AMBARI-18736
> https://issues.apache.org/jira/browse/AMBARI-18736
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The PERF stack can actually create Python scripts to run actual alerts whose behavior is controlled by configs to either always pass, always fail, pass with some success %, flip/thrash, timeout, etc.
>
> For other types of alerts (web, host:port), we will need to override the Alert Scheduler on the Ambari Agent to return fake values.
>
> The goal is to stress test Ambari Server when multiple Ambari Agents run on a single host.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/metainfo.xml 36a2168
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/package/alerts/alert_happy_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/alerts.json 1b3ae25
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metainfo.xml b304706
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_master_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_regionserver_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/alerts.json 8ccfa47
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/metainfo.xml bf26fbd
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_checkpoint_time.py 1e69def
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_datanode_unmounted_data_dir.py 8c122ae
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_ha_namenode_health.py 7f03d88
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_metrics_deviation.py 0946c85
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_nfs_gateway_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_snamenode_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_upgrade_finalized.py 0dff200
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/configuration/sleepy-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/metainfo.xml 470d809
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/package/alerts/alert_sleepy_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/configuration/snow-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/metainfo.xml cbb57f6
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/package/alerts/alert_snow_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/alerts.json c4a58bb
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/metainfo.xml 8b18e73
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_history_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanager_health.py d2cfe26
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanagers_summary.py 81527f6
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_resourcemanager_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_timeline_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/configuration/zk-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/metainfo.xml 62ea041
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/package/alerts/alert_zk_server_process.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/54187/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 54187: Perf: Simulate alerts for multiple Ambari
Agents running on single Host
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54187/#review157495
-----------------------------------------------------------
Fix it, then Ship it!
ambari-common/src/main/python/resource_management/libraries/functions/simulate_perf_cluster_alert_behaviour.py (line 42)
<https://reviews.apache.org/r/54187/#comment228100>
I prefer not to have
alert_behaviour_type_key, alert_success_percentage_key, alert_timeout_return_value_key,
alert_timeout_secs_key, alert_flip_interval_mins_key
as individual parameters but rather put them in a dictionary so that it's easier for the callers.
If we ever add more configs here, it will be easier so we don't have to change a lot of files.
- Alejandro Fernandez
On Nov. 30, 2016, 11:29 p.m., Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54187/
> -----------------------------------------------------------
>
> (Updated Nov. 30, 2016, 11:29 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Sid Wagle.
>
>
> Bugs: AMBARI-18736
> https://issues.apache.org/jira/browse/AMBARI-18736
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The PERF stack can actually create Python scripts to run actual alerts whose behavior is controlled by configs to either always pass, always fail, pass with some success %, flip/thrash, timeout, etc.
>
> For other types of alerts (web, host:port), we will need to override the Alert Scheduler on the Ambari Agent to return fake values.
>
> The goal is to stress test Ambari Server when multiple Ambari Agents run on a single host.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/simulate_perf_cluster_alert_behaviour.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/metainfo.xml 36a2168
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/package/alerts/alert_happy_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/alerts.json 1b3ae25
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metainfo.xml b304706
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_master_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_regionserver_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/alerts.json 8ccfa47
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/metainfo.xml bf26fbd
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_checkpoint_time.py 1e69def
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_datanode_unmounted_data_dir.py 8c122ae
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_ha_namenode_health.py 7f03d88
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_metrics_deviation.py 0946c85
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_nfs_gateway_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_snamenode_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_upgrade_finalized.py 0dff200
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/configuration/sleepy-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/metainfo.xml 470d809
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/package/alerts/alert_sleepy_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/configuration/snow-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/metainfo.xml cbb57f6
> ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/package/alerts/alert_snow_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/alerts.json c4a58bb
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/metainfo.xml 8b18e73
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_history_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanager_health.py d2cfe26
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanagers_summary.py 81527f6
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_resourcemanager_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_timeline_process.py PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/alerts.json PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/configuration/zk-alert-config.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/metainfo.xml 62ea041
> ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/package/alerts/alert_zk_server_process.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/54187/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 54187: Perf: Simulate alerts for multiple Ambari
Agents running on single Host
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54187/
-----------------------------------------------------------
(Updated \u0413\u0440\u0443. 1, 2016, 1:51 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f)
Review request for Ambari, Alejandro Fernandez and Sid Wagle.
Bugs: AMBARI-18736
https://issues.apache.org/jira/browse/AMBARI-18736
Repository: ambari
Description
-------
The PERF stack can actually create Python scripts to run actual alerts whose behavior is controlled by configs to either always pass, always fail, pass with some success %, flip/thrash, timeout, etc.
For other types of alerts (web, host:port), we will need to override the Alert Scheduler on the Ambari Agent to return fake values.
The goal is to stress test Ambari Server when multiple Ambari Agents run on a single host.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/simulate_perf_cluster_alert_behaviour.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/metainfo.xml 36a2168
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/package/alerts/alert_happy_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/alerts.json 1b3ae25
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metainfo.xml b304706
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_master_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_regionserver_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/alerts.json 8ccfa47
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/metainfo.xml bf26fbd
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_checkpoint_time.py 1e69def
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_datanode_unmounted_data_dir.py 8c122ae
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_ha_namenode_health.py 7f03d88
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_metrics_deviation.py 0946c85
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_nfs_gateway_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_snamenode_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_upgrade_finalized.py 0dff200
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/configuration/sleepy-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/metainfo.xml 470d809
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/package/alerts/alert_sleepy_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/configuration/snow-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/metainfo.xml cbb57f6
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/package/alerts/alert_snow_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/alerts.json c4a58bb
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/metainfo.xml 8b18e73
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_history_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanager_health.py d2cfe26
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanagers_summary.py 81527f6
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_resourcemanager_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_timeline_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/configuration/zk-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/metainfo.xml 62ea041
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/package/alerts/alert_zk_server_process.py PRE-CREATION
Diff: https://reviews.apache.org/r/54187/diff/
Testing
-------
mvn clean test
Thanks,
Vitalyi Brodetskyi
Re: Review Request 54187: Perf: Simulate alerts for multiple Ambari
Agents running on single Host
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54187/
-----------------------------------------------------------
(Updated \u041b\u0438\u0441. 30, 2016, 11:29 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f)
Review request for Ambari, Alejandro Fernandez and Sid Wagle.
Bugs: AMBARI-18736
https://issues.apache.org/jira/browse/AMBARI-18736
Repository: ambari
Description
-------
The PERF stack can actually create Python scripts to run actual alerts whose behavior is controlled by configs to either always pass, always fail, pass with some success %, flip/thrash, timeout, etc.
For other types of alerts (web, host:port), we will need to override the Alert Scheduler on the Ambari Agent to return fake values.
The goal is to stress test Ambari Server when multiple Ambari Agents run on a single host.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/simulate_perf_cluster_alert_behaviour.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/configuration/happy-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/metainfo.xml 36a2168
ambari-server/src/main/resources/stacks/PERF/1.0/services/HAPPY/package/alerts/alert_happy_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/alerts.json 1b3ae25
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/configuration/hbase-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/metainfo.xml b304706
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_master_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HBASE/package/alerts/hbase_regionserver_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/alerts.json 8ccfa47
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/configuration/hdfs-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/metainfo.xml bf26fbd
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_checkpoint_time.py 1e69def
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_datanode_unmounted_data_dir.py 8c122ae
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_ha_namenode_health.py 7f03d88
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_metrics_deviation.py 0946c85
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_nfs_gateway_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_snamenode_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/HDFS/package/alerts/alert_upgrade_finalized.py 0dff200
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/configuration/sleepy-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/metainfo.xml 470d809
ambari-server/src/main/resources/stacks/PERF/1.0/services/SLEEPY/package/alerts/alert_sleepy_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/configuration/snow-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/metainfo.xml cbb57f6
ambari-server/src/main/resources/stacks/PERF/1.0/services/SNOW/package/alerts/alert_snow_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/alerts.json c4a58bb
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/configuration/yarn-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/metainfo.xml 8b18e73
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_history_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanager_health.py d2cfe26
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_nodemanagers_summary.py 81527f6
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_resourcemanager_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/YARN/package/alerts/alert_timeline_process.py PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/alerts.json PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/configuration/zk-alert-config.xml PRE-CREATION
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/metainfo.xml 62ea041
ambari-server/src/main/resources/stacks/PERF/1.0/services/ZOOKEEPER/package/alerts/alert_zk_server_process.py PRE-CREATION
Diff: https://reviews.apache.org/r/54187/diff/
Testing
-------
mvn clean test
Thanks,
Vitalyi Brodetskyi