You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Tim Thorpe <tt...@ca.ibm.com> on 2016/03/01 17:53:49 UTC
Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/
-----------------------------------------------------------
Review request for Ambari, Sumit Mohanty and Yusaku Sako.
Bugs: AMBARI-15226
https://issues.apache.org/jira/browse/AMBARI-15226
Repository: ambari
Description
-------
Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
Diffs
-----
ambari-agent/pom.xml c2c993f
ambari-server/src/main/assemblies/server.xml e1a4919
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
Diff: https://reviews.apache.org/r/44210/diff/
Testing
-------
Ran all the non java unit tests.
Total run:921
Total errors:1
Total failures:0
The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
Thanks,
Tim Thorpe
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On March 2, 2016, 9:37 p.m., jun aoki wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py, line 1
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275516#file1275516line1>
> >
> > Hi Tim, nice effort to decouple stack_advisor!
> >
> > Is this file naming service_advisor_HDP206HBASE.py a bit redundant? this is under ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE, any files under this folder is meant for HDP, version 2.0.6 and HBASE.
I tried having the name as you stated but had a problem with python. For example in 2.3/services/HDFS, I have the following:
SCRIPT_DIR = os.path.dirname(os.path.abspath(file))
PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/HDFS/')
sys.path.append(PARENT_DIR)
from service_advisor_HDP22HDFS import *
If the file name is the same in 2.3 and 2.2, then "from service_advisor import *" could reference the 2.2 or the 2.3 file. I tried it with import HDP22HDFSServiceAdvisor as well and both of them failed to find the correct parent class.
The other way I could have solved this is to load all the service advisor files from the default_stack_advisor.py walking my way up the stack. This would have allowed you to avoid the inheritance code in the service_advisor.py files but it would restrict you from inheriting from outside of your stack (unless at that point you started including the inheritance code and the change of file names).
> On March 2, 2016, 9:37 p.m., jun aoki wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py, line 32
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275516#file1275516line32>
> >
> > Could this inheritate a brand new DefaultServiceAdvisor instead of the stack advisor? service and stack are different hierarchy and I feel somehow wrong if a service inheritate a stack.
Point well taken. I'll look at refactoring it that way but really would prefer to get the code in the current way first. There are code changes to the stack advisor files on a daily basis and the longer it waits the more merges will be needed. I will already be refactoring the code to split the test scripts to be on the service level rather than the stack level. I will attempt to do both of these tasks at the same time.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121720
-----------------------------------------------------------
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Matt <mi...@gmail.com>.
> On March 2, 2016, 1:37 p.m., jun aoki wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py, line 32
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275516#file1275516line32>
> >
> > Could this inheritate a brand new DefaultServiceAdvisor instead of the stack advisor? service and stack are different hierarchy and I feel somehow wrong if a service inheritate a stack.
>
> Tim Thorpe wrote:
> Point well taken. I'll look at refactoring it that way but really would prefer to get the code in the current way first. There are code changes to the stack advisor files on a daily basis and the longer it waits the more merges will be needed. I will already be refactoring the code to split the test scripts to be on the service level rather than the stack level. I will attempt to do both of these tasks at the same time.
May be this should inherit from service_advisor for that service under common-services?
- Matt
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121720
-----------------------------------------------------------
On April 12, 2016, 10:33 a.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 12, 2016, 10:33 a.m.)
>
>
> Review request for Ambari, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On March 2, 2016, 9:37 p.m., jun aoki wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py, line 1
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275516#file1275516line1>
> >
> > Hi Tim, nice effort to decouple stack_advisor!
> >
> > Is this file naming service_advisor_HDP206HBASE.py a bit redundant? this is under ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE, any files under this folder is meant for HDP, version 2.0.6 and HBASE.
>
> Tim Thorpe wrote:
> I tried having the name as you stated but had a problem with python. For example in 2.3/services/HDFS, I have the following:
>
> SCRIPT_DIR = os.path.dirname(os.path.abspath(file))
> PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/HDFS/')
> sys.path.append(PARENT_DIR)
> from service_advisor_HDP22HDFS import *
>
> If the file name is the same in 2.3 and 2.2, then "from service_advisor import *" could reference the 2.2 or the 2.3 file. I tried it with import HDP22HDFSServiceAdvisor as well and both of them failed to find the correct parent class.
>
> The other way I could have solved this is to load all the service advisor files from the default_stack_advisor.py walking my way up the stack. This would have allowed you to avoid the inheritance code in the service_advisor.py files but it would restrict you from inheriting from outside of your stack (unless at that point you started including the inheritance code and the change of file names).
>
> jun aoki wrote:
> I believe it has to treated as python packages so that file path matters.
> ```
> sys.path.append("..../stacks")
> from HDP.2_2.services.HDFS import service_advisor # you'd probably have to deal with "." in versions.
> ```
I have figured out a solution for this. So the naming of the service advisor file would always be service_advisor.py without being prefaced with the stack name, version and service name.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121720
-----------------------------------------------------------
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On March 2, 2016, 9:37 p.m., jun aoki wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py, line 32
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275516#file1275516line32>
> >
> > Could this inheritate a brand new DefaultServiceAdvisor instead of the stack advisor? service and stack are different hierarchy and I feel somehow wrong if a service inheritate a stack.
>
> Tim Thorpe wrote:
> Point well taken. I'll look at refactoring it that way but really would prefer to get the code in the current way first. There are code changes to the stack advisor files on a daily basis and the longer it waits the more merges will be needed. I will already be refactoring the code to split the test scripts to be on the service level rather than the stack level. I will attempt to do both of these tasks at the same time.
>
> Matt wrote:
> May be this should inherit from service_advisor for that service under common-services?
Hi Matt, so you are recommending that the resources/stacks/service_advisor.py be moved to resources/common-services/service_advisor.py?
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121720
-----------------------------------------------------------
On April 12, 2016, 5:33 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 12, 2016, 5:33 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by jun aoki <ju...@gmail.com>.
> On March 2, 2016, 9:37 p.m., jun aoki wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py, line 1
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275516#file1275516line1>
> >
> > Hi Tim, nice effort to decouple stack_advisor!
> >
> > Is this file naming service_advisor_HDP206HBASE.py a bit redundant? this is under ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE, any files under this folder is meant for HDP, version 2.0.6 and HBASE.
>
> Tim Thorpe wrote:
> I tried having the name as you stated but had a problem with python. For example in 2.3/services/HDFS, I have the following:
>
> SCRIPT_DIR = os.path.dirname(os.path.abspath(file))
> PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/HDFS/')
> sys.path.append(PARENT_DIR)
> from service_advisor_HDP22HDFS import *
>
> If the file name is the same in 2.3 and 2.2, then "from service_advisor import *" could reference the 2.2 or the 2.3 file. I tried it with import HDP22HDFSServiceAdvisor as well and both of them failed to find the correct parent class.
>
> The other way I could have solved this is to load all the service advisor files from the default_stack_advisor.py walking my way up the stack. This would have allowed you to avoid the inheritance code in the service_advisor.py files but it would restrict you from inheriting from outside of your stack (unless at that point you started including the inheritance code and the change of file names).
I believe it has to treated as python packages so that file path matters.
```
sys.path.append("..../stacks")
from HDP.2_2.services.HDFS import service_advisor # you'd probably have to deal with "." in versions.
```
- jun
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121720
-----------------------------------------------------------
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On March 2, 2016, 9:37 p.m., jun aoki wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py, line 32
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275516#file1275516line32>
> >
> > Could this inheritate a brand new DefaultServiceAdvisor instead of the stack advisor? service and stack are different hierarchy and I feel somehow wrong if a service inheritate a stack.
>
> Tim Thorpe wrote:
> Point well taken. I'll look at refactoring it that way but really would prefer to get the code in the current way first. There are code changes to the stack advisor files on a daily basis and the longer it waits the more merges will be needed. I will already be refactoring the code to split the test scripts to be on the service level rather than the stack level. I will attempt to do both of these tasks at the same time.
>
> Matt wrote:
> May be this should inherit from service_advisor for that service under common-services?
>
> Tim Thorpe wrote:
> Hi Matt, so you are recommending that the resources/stacks/service_advisor.py be moved to resources/common-services/service_advisor.py?
Probably a more logical place to put it would be under resources/scripts
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121720
-----------------------------------------------------------
On April 12, 2016, 5:33 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 12, 2016, 5:33 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by jun aoki <ju...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121720
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py (line 1)
<https://reviews.apache.org/r/44210/#comment183508>
Hi Tim, nice effort to decouple stack_advisor!
Is this file naming service_advisor_HDP206HBASE.py a bit redundant? this is under ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE, any files under this folder is meant for HDP, version 2.0.6 and HBASE.
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py (line 32)
<https://reviews.apache.org/r/44210/#comment183517>
Could this inheritate a brand new DefaultServiceAdvisor instead of the stack advisor? service and stack are different hierarchy and I feel somehow wrong if a service inheritate a stack.
- jun aoki
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 5, 2016, 5:32 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py, line 1
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275506#file1275506line1>
> >
> > Naming each service_advisor script named with stackname-version-servicename doesnt quite feel elegant. Simply service_advisor.py is what should be there at service-level.
> >
> > Also, we should do cross-stack references. When we bundle a stack in an Ambari release, we only bundle one-stack. So for BIGTOP, the HDP stack directory will not be present. Otherwise if you include both HDP and BIGTOP in the Ambari release, in the UI, you will see both BIGTOP and HDP stack options.
I have figured out a solution for this. So the naming of the service advisor file would always be service_advisor.py without being prefaced with the stack name, version and service name.
Since the stack advisors are no longer being refactored, we will only allow for service advisors to be plugged in, there will be no dependence of BIGTOP to HDP.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review127148
-----------------------------------------------------------
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review127148
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py (line 1)
<https://reviews.apache.org/r/44210/#comment190286>
Naming each service_advisor script named with stackname-version-servicename doesnt quite feel elegant. Simply service_advisor.py is what should be there at service-level.
Also, we should do cross-stack references. When we bundle a stack in an Ambari release, we only bundle one-stack. So for BIGTOP, the HDP stack directory will not be present. Otherwise if you include both HDP and BIGTOP in the Ambari release, in the UI, you will see both BIGTOP and HDP stack options.
- Jayush Luniya
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 22, 2016, 5:15 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/service_advisor.py, line 24
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348349#file1348349line24>
> >
> > Add documentation for all functions here.
I will make sure I document all the functions in the service advisor.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130124
-----------------------------------------------------------
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130124
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/service_advisor.py (line 24)
<https://reviews.apache.org/r/44210/#comment193786>
Add documentation for all functions here.
- Jayush Luniya
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 22, 2016, 6:15 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py, line 31
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348345#file1348345line31>
> >
> > Lets say we have
> > common-services/MYSERVICE/1.0.0
> > |__ service_advisor.py
> > common-services/MYSERVICE/2.0.0 (<- extends common-services/MYSERVICE/1.0.0)
> > |__ service_advisor.py
> >
> > stacks/HDP/2.3/services/MYSERVICE (<- extends common-services/MYSERVICE/1.0.0)
> > |__ service_advisor.py
> >
> > stacks/HDP/2.5/services/MYSERVICE (<- extends common-services/MYSERVICE/2.0.0)
> > |__ service_advisor.py
> >
> > I believe we wont support any sort of inheritance between them? I am ok with keeping things simple right now but we should call out this limitation and that there could be certain duplication.
>
> Jayush Luniya wrote:
> Sorry the formatting went off after publishing :(
>
> Tim Thorpe wrote:
> Hi Jayush, you are right there won't be any automatic inheritance but I don't think that's really a bad thing. You can still have explicit inheritance with code like this:
>
> SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
> PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/YARN/')
> PARENT_FILE = os.path.join(PARENT_DIR, 'service_advisor.py')
>
> try:
> with open(PARENT_FILE, 'rb') as fp:
> service_advisor = imp.load_module('service_advisor', fp, PARENT_FILE, ('.py', 'rb', imp.PY_SOURCE))
> except Exception as e:
> traceback.print_exc()
> print "Failed to load parent"
>
>
> class HDP23MAPREDUCE2ServiceAdvisor(service_advisor.HDP22MAPREDUCE2ServiceAdvisor):
>
> Jayush Luniya wrote:
> Lets just document this in the abstract service_advisor.py. Since stack advisor supports such inheritance, just want to make sure that we call this out and devs dont assume it will just work :)
I'll add that in when I do my documentation changes.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130141
-----------------------------------------------------------
On April 22, 2016, 6:27 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 6:27 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
> On April 22, 2016, 6:15 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py, line 31
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348345#file1348345line31>
> >
> > Lets say we have
> > common-services/MYSERVICE/1.0.0
> > |__ service_advisor.py
> > common-services/MYSERVICE/2.0.0 (<- extends common-services/MYSERVICE/1.0.0)
> > |__ service_advisor.py
> >
> > stacks/HDP/2.3/services/MYSERVICE (<- extends common-services/MYSERVICE/1.0.0)
> > |__ service_advisor.py
> >
> > stacks/HDP/2.5/services/MYSERVICE (<- extends common-services/MYSERVICE/2.0.0)
> > |__ service_advisor.py
> >
> > I believe we wont support any sort of inheritance between them? I am ok with keeping things simple right now but we should call out this limitation and that there could be certain duplication.
Sorry the formatting went off after publishing :(
- Jayush
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130141
-----------------------------------------------------------
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Alexander Denissov <ad...@apache.org>.
> On April 22, 2016, 6:15 p.m., Jayush Luniya wrote:
> >
So, what will be the rules for SA logic inheritance ?
When MYSERVICE/2.0.0 ships and needs to reuse the stack_advisor logic from MYSERVICE/1.0.0 -- will we:
- rely on stack inheritance ?
- rely on SA inheritance within MYSERVICE ?
- combination of these ?
- Alexander
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130141
-----------------------------------------------------------
On April 22, 2016, 6:27 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 6:27 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 22, 2016, 6:15 p.m., Jayush Luniya wrote:
> >
>
> Alexander Denissov wrote:
> So, what will be the rules for SA logic inheritance ?
>
> When MYSERVICE/2.0.0 ships and needs to reuse the stack_advisor logic from MYSERVICE/1.0.0 -- will we:
> - rely on stack inheritance ?
> - rely on SA inheritance within MYSERVICE ?
> - combination of these ?
There will be no implicit inheritance. If you want to use inheritance you can but it will be left up to the service to handle it with code like:
SCRIPT_DIR = os.path.dirname(os.path.abspath(file))
PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/YARN/')
PARENT_FILE = os.path.join(PARENT_DIR, 'service_advisor.py')
try:
with open(PARENT_FILE, 'rb') as fp:
service_advisor = imp.load_module('service_advisor', fp, PARENT_FILE, ('.py', 'rb', imp.PY_SOURCE))
except Exception as e:
traceback.print_exc()
print "Failed to load parent"
class HDP23MAPREDUCE2ServiceAdvisor(service_advisor.HDP22MAPREDUCE2ServiceAdvisor):
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130141
-----------------------------------------------------------
On April 22, 2016, 6:27 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 6:27 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 22, 2016, 6:15 p.m., Jayush Luniya wrote:
> >
>
> Alexander Denissov wrote:
> So, what will be the rules for SA logic inheritance ?
>
> When MYSERVICE/2.0.0 ships and needs to reuse the stack_advisor logic from MYSERVICE/1.0.0 -- will we:
> - rely on stack inheritance ?
> - rely on SA inheritance within MYSERVICE ?
> - combination of these ?
>
> Tim Thorpe wrote:
> There will be no implicit inheritance. If you want to use inheritance you can but it will be left up to the service to handle it with code like:
>
> SCRIPT_DIR = os.path.dirname(os.path.abspath(file))
> PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/YARN/')
> PARENT_FILE = os.path.join(PARENT_DIR, 'service_advisor.py')
>
> try:
> with open(PARENT_FILE, 'rb') as fp:
> service_advisor = imp.load_module('service_advisor', fp, PARENT_FILE, ('.py', 'rb', imp.PY_SOURCE))
> except Exception as e:
> traceback.print_exc()
> print "Failed to load parent"
>
>
> class HDP23MAPREDUCE2ServiceAdvisor(service_advisor.HDP22MAPREDUCE2ServiceAdvisor):
So I guess the answer to your question is none of those.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130141
-----------------------------------------------------------
On April 22, 2016, 6:27 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 6:27 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 22, 2016, 6:15 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py, line 31
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348345#file1348345line31>
> >
> > Lets say we have
> > common-services/MYSERVICE/1.0.0
> > |__ service_advisor.py
> > common-services/MYSERVICE/2.0.0 (<- extends common-services/MYSERVICE/1.0.0)
> > |__ service_advisor.py
> >
> > stacks/HDP/2.3/services/MYSERVICE (<- extends common-services/MYSERVICE/1.0.0)
> > |__ service_advisor.py
> >
> > stacks/HDP/2.5/services/MYSERVICE (<- extends common-services/MYSERVICE/2.0.0)
> > |__ service_advisor.py
> >
> > I believe we wont support any sort of inheritance between them? I am ok with keeping things simple right now but we should call out this limitation and that there could be certain duplication.
>
> Jayush Luniya wrote:
> Sorry the formatting went off after publishing :(
Hi Jayush, you are right there won't be any automatic inheritance but I don't think that's really a bad thing. You can still have explicit inheritance with code like this:
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/YARN/')
PARENT_FILE = os.path.join(PARENT_DIR, 'service_advisor.py')
try:
with open(PARENT_FILE, 'rb') as fp:
service_advisor = imp.load_module('service_advisor', fp, PARENT_FILE, ('.py', 'rb', imp.PY_SOURCE))
except Exception as e:
traceback.print_exc()
print "Failed to load parent"
class HDP23MAPREDUCE2ServiceAdvisor(service_advisor.HDP22MAPREDUCE2ServiceAdvisor):
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130141
-----------------------------------------------------------
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
> On April 22, 2016, 6:15 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py, line 31
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348345#file1348345line31>
> >
> > Lets say we have
> > common-services/MYSERVICE/1.0.0
> > |__ service_advisor.py
> > common-services/MYSERVICE/2.0.0 (<- extends common-services/MYSERVICE/1.0.0)
> > |__ service_advisor.py
> >
> > stacks/HDP/2.3/services/MYSERVICE (<- extends common-services/MYSERVICE/1.0.0)
> > |__ service_advisor.py
> >
> > stacks/HDP/2.5/services/MYSERVICE (<- extends common-services/MYSERVICE/2.0.0)
> > |__ service_advisor.py
> >
> > I believe we wont support any sort of inheritance between them? I am ok with keeping things simple right now but we should call out this limitation and that there could be certain duplication.
>
> Jayush Luniya wrote:
> Sorry the formatting went off after publishing :(
>
> Tim Thorpe wrote:
> Hi Jayush, you are right there won't be any automatic inheritance but I don't think that's really a bad thing. You can still have explicit inheritance with code like this:
>
> SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
> PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/YARN/')
> PARENT_FILE = os.path.join(PARENT_DIR, 'service_advisor.py')
>
> try:
> with open(PARENT_FILE, 'rb') as fp:
> service_advisor = imp.load_module('service_advisor', fp, PARENT_FILE, ('.py', 'rb', imp.PY_SOURCE))
> except Exception as e:
> traceback.print_exc()
> print "Failed to load parent"
>
>
> class HDP23MAPREDUCE2ServiceAdvisor(service_advisor.HDP22MAPREDUCE2ServiceAdvisor):
Lets just document this in the abstract service_advisor.py. Since stack advisor supports such inheritance, just want to make sure that we call this out and devs dont assume it will just work :)
- Jayush
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130141
-----------------------------------------------------------
On April 22, 2016, 6:27 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 6:27 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130141
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py (line 31)
<https://reviews.apache.org/r/44210/#comment193816>
Lets say we have
common-services/MYSERVICE/1.0.0
|__ service_advisor.py
common-services/MYSERVICE/2.0.0 (<- extends common-services/MYSERVICE/1.0.0)
|__ service_advisor.py
stacks/HDP/2.3/services/MYSERVICE (<- extends common-services/MYSERVICE/1.0.0)
|__ service_advisor.py
stacks/HDP/2.5/services/MYSERVICE (<- extends common-services/MYSERVICE/2.0.0)
|__ service_advisor.py
I believe we wont support any sort of inheritance between them? I am ok with keeping things simple right now but we should call out this limitation and that there could be certain duplication.
- Jayush Luniya
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 22, 2016, 5:10 p.m., Srimanth Gunturi wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java, line 77
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348336#file1348336line77>
> >
> > Trying to think if we really need to expose the 'advisor_name' and 'advisor_path' in the stack-service API response? The files will be at known locations anyways and used when available... similar to the stack's advisor_path/advisor_name.
> >
> > I am thinking this can work without adding these 2 properties to a stack-service.
>
> Tim Thorpe wrote:
> The reason this works for the stack is because the stack_advisor.py in the scripts directory attempts to load all the stack_advisor.py files. It uses the stack name, stack version and parent versions to determine what to load. It starts with the oldest version and keeps loading until the most recent version. In order to do this for services, we'd need to know the some sort of information to determine what to load from where. I could have added something similar so that it would look into the stack versions and then somehow look to common services if that was required. In my code I was testing with HAWQ and PXF with their service advisor's being loaded from the common-services directory. It seemed much easier to me at least to get the service_advisor.py location and calculate the advisor name in the java code when reading the stack and services (StackManager/StackModule/StackDirectory etc...). This way I know exactly where to load the py file and exactly what class name t
o use.
>
> Srimanth Gunturi wrote:
> Yes, I agree that it makes it easy to determine location of files. But if you look at it from pure API perspective, it is not useful for any caller except ambari-server - None of the callers have access to those paths/files except ambari-server. It is ambari-server's internals exposed outside just for its own consumption. Also, like the stack's service-advisor.py, the service's files can be dynamically located and loaded.
>
> If it is a question of efficiency, we can cache the result so that for a stack-version's service we only determine this once.
>
> My vote would be to not have these properties on the stack service's response.
Instead of adding them to the stack service's response, I can inject it only for the stack advisor calls. I'll work on making the change.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130123
-----------------------------------------------------------
On April 22, 2016, 6:27 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 6:27 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 22, 2016, 5:10 p.m., Srimanth Gunturi wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java, line 77
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348336#file1348336line77>
> >
> > Trying to think if we really need to expose the 'advisor_name' and 'advisor_path' in the stack-service API response? The files will be at known locations anyways and used when available... similar to the stack's advisor_path/advisor_name.
> >
> > I am thinking this can work without adding these 2 properties to a stack-service.
The reason this works for the stack is because the stack_advisor.py in the scripts directory attempts to load all the stack_advisor.py files. It uses the stack name, stack version and parent versions to determine what to load. It starts with the oldest version and keeps loading until the most recent version. In order to do this for services, we'd need to know the some sort of information to determine what to load from where. I could have added something similar so that it would look into the stack versions and then somehow look to common services if that was required. In my code I was testing with HAWQ and PXF with their service advisor's being loaded from the common-services directory. It seemed much easier to me at least to get the service_advisor.py location and calculate the advisor name in the java code when reading the stack and services (StackManager/StackModule/StackDirectory etc...). This way I know exactly where to load the py file and exactly what class name to use.
> On April 22, 2016, 5:10 p.m., Srimanth Gunturi wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java, line 88
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348338#file1348338line88>
> >
> > I think we can make this work without adding new properties to stack-service resource.
see my comment above.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130123
-----------------------------------------------------------
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Srimanth Gunturi <sr...@hortonworks.com>.
> On April 22, 2016, 5:10 p.m., Srimanth Gunturi wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java, line 77
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348336#file1348336line77>
> >
> > Trying to think if we really need to expose the 'advisor_name' and 'advisor_path' in the stack-service API response? The files will be at known locations anyways and used when available... similar to the stack's advisor_path/advisor_name.
> >
> > I am thinking this can work without adding these 2 properties to a stack-service.
>
> Tim Thorpe wrote:
> The reason this works for the stack is because the stack_advisor.py in the scripts directory attempts to load all the stack_advisor.py files. It uses the stack name, stack version and parent versions to determine what to load. It starts with the oldest version and keeps loading until the most recent version. In order to do this for services, we'd need to know the some sort of information to determine what to load from where. I could have added something similar so that it would look into the stack versions and then somehow look to common services if that was required. In my code I was testing with HAWQ and PXF with their service advisor's being loaded from the common-services directory. It seemed much easier to me at least to get the service_advisor.py location and calculate the advisor name in the java code when reading the stack and services (StackManager/StackModule/StackDirectory etc...). This way I know exactly where to load the py file and exactly what class name t
o use.
Yes, I agree that it makes it easy to determine location of files. But if you look at it from pure API perspective, it is not useful for any caller except ambari-server - None of the callers have access to those paths/files except ambari-server. It is ambari-server's internals exposed outside just for its own consumption. Also, like the stack's service-advisor.py, the service's files can be dynamically located and loaded.
If it is a question of efficiency, we can cache the result so that for a stack-version's service we only determine this once.
My vote would be to not have these properties on the stack service's response.
- Srimanth
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130123
-----------------------------------------------------------
On April 22, 2016, 6:27 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 6:27 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Srimanth Gunturi <sr...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130123
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java (line 77)
<https://reviews.apache.org/r/44210/#comment193784>
Trying to think if we really need to expose the 'advisor_name' and 'advisor_path' in the stack-service API response? The files will be at known locations anyways and used when available... similar to the stack's advisor_path/advisor_name.
I am thinking this can work without adding these 2 properties to a stack-service.
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java (line 88)
<https://reviews.apache.org/r/44210/#comment193785>
I think we can make this work without adding new properties to stack-service resource.
- Srimanth Gunturi
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
> On April 22, 2016, 5:33 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/stack_advisor.py, line 522
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348350#file1348350line522>
> >
> > loadServiceAdvisor() instead?
>
> Tim Thorpe wrote:
> If I follow the tradition of the scripts/stack_advisor.py, I guess I should call it def instantiateServiceAdvisor(). Any one of those names are fine by me. create or instantiate are more accurate names I think, because it actually does both load the script and create an instance.
instantiateServiceAdvisor() is fine.
- Jayush
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130131
-----------------------------------------------------------
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 22, 2016, 5:33 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/stack_advisor.py, line 522
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348350#file1348350line522>
> >
> > loadServiceAdvisor() instead?
If I follow the tradition of the scripts/stack_advisor.py, I guess I should call it def instantiateServiceAdvisor(). Any one of those names are fine by me. create or instantiate are more accurate names I think, because it actually does both load the script and create an instance.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130131
-----------------------------------------------------------
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130131
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/stack_advisor.py (line 507)
<https://reviews.apache.org/r/44210/#comment193792>
loadServiceAdvisor() instead?
- Jayush Luniya
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 22, 2016, 5:58 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/stack_advisor.py, line 473
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348350#file1348350line473>
> >
> > I might be missing something, but it doesnt look like we are using getHostsForSlaveComponent
Yep I missed something in that patch, when I updated an older patch over some later code. The createComponentLayoutRecommendations code should have the client/slave section like this:
for service in services["services"]:
slaveClientComponents = [component for component in service["components"]
if self.isSlaveComponent(component) or self.isClientComponent(component)]
serviceAdvisor = self.createServiceAdvisor(service)
for component in slaveClientComponents:
componentName = component["StackServiceComponents"]["component_name"]
hostsForComponent = []
if serviceAdvisor is None:
hostsForComponent = self.getHostsForSlaveComponent(services, hosts, component, hostsList, hostsComponentsMap)
else:
hostsForComponent = serviceAdvisor.getHostsForSlaveComponent(self, services, hosts, component, hostsList, hostsComponentsMap)
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130138
-----------------------------------------------------------
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130138
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/stack_advisor.py (line 458)
<https://reviews.apache.org/r/44210/#comment193803>
I might be missing something, but it doesnt look like we are using getHostsForSlaveComponent
- Jayush Luniya
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
> On April 22, 2016, 5:18 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py, line 19
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348344#file1348344line19>
> >
> > Please add Pivotal folks to the review to look at this. Also since the patch is a bit old, please verify and revise the patch to include any latest changes.
>
> Tim Thorpe wrote:
> I will update the patch to the latest once I get all the feedback including the Pivotal folks. Otherwise, I'm going to be updating the patch many times before I get to the point where it can be accept for commit. Can you please let me know who I should add from Pivotal? Thanks
https://reviews.apache.org/users/bhuvnesh2703/
https://reviews.apache.org/users/adenissov/
https://reviews.apache.org/users/odiachenko/
- Jayush
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130126
-----------------------------------------------------------
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 22, 2016, 5:18 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py, line 19
> > <https://reviews.apache.org/r/44210/diff/2/?file=1348344#file1348344line19>
> >
> > Please add Pivotal folks to the review to look at this. Also since the patch is a bit old, please verify and revise the patch to include any latest changes.
I will update the patch to the latest once I get all the feedback including the Pivotal folks. Otherwise, I'm going to be updating the patch many times before I get to the point where it can be accept for commit. Can you please let me know who I should add from Pivotal? Thanks
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130126
-----------------------------------------------------------
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130126
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 19)
<https://reviews.apache.org/r/44210/#comment193788>
Please add Pivotal folks to the review to look at this. Also since the patch is a bit old, please verify and revise the patch to include any latest changes.
- Jayush Luniya
On April 22, 2016, 4:39 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 22, 2016, 4:39 p.m.)
>
>
> Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
> ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/
-----------------------------------------------------------
(Updated April 22, 2016, 4:39 p.m.)
Review request for Ambari, Jayush Luniya, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
Bugs: AMBARI-15226
https://issues.apache.org/jira/browse/AMBARI-15226
Repository: ambari
Description
-------
Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
Diffs (updated)
-----
ambari-server/src/main/assemblies/server.xml e1a4919
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java df65010
ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java 00c8696
ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java ca1968e
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java 6c6fa91
ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java 636de37
ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java d574d60
ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java b7e09a9
ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java d27e52a
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java db95fec
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
ambari-server/src/main/resources/properties.json eac0dbd
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 0130483
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 3a65541
ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION
ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 6c9fd46
Diff: https://reviews.apache.org/r/44210/diff/
Testing
-------
Ran all the non java unit tests.
Total run:945
Total errors:0
Total failures:0
Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
Thanks,
Tim Thorpe
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/
-----------------------------------------------------------
(Updated April 12, 2016, 5:33 p.m.)
Review request for Ambari, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
Bugs: AMBARI-15226
https://issues.apache.org/jira/browse/AMBARI-15226
Repository: ambari
Description
-------
Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
Diffs
-----
ambari-agent/pom.xml c2c993f
ambari-server/src/main/assemblies/server.xml e1a4919
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
Diff: https://reviews.apache.org/r/44210/diff/
Testing (updated)
-------
Ran all the non java unit tests.
Total run:945
Total errors:0
Total failures:0
Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
Thanks,
Tim Thorpe
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review127144
-----------------------------------------------------------
I dont think we should go down the route of breaking up the stack_advisor from stack-level to service-level at one shot. It is too risky a change. Instead we should support service-level advisors as extensions to stack_advisors so that add-on/custom services can extend the stack_advisors. Similar to how you handled RCO.
- Jayush Luniya
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On April 5, 2016, 7:53 p.m., Srimanth Gunturi wrote:
> > Tim, thanks for contributing to the stack-advisor feature.
> >
> > When we started the stack-advisor feature, there was a lot of interconnect between base service component-layout and configurations (HDFS, YARN, MapReduce, Hive, HBase) where common calculations were used by multiple services. To maximise readability and reuse of code, we took the 1 stack-advisor/stack-version approach. This has kept the number of scripts managable (8 scripts for all versions of HDP stack) to maintain and debug when problems occur. It is very useful for stacks where a lot of interconnected services exist and stack-advisor should account for all the usecases.
> >
> > Now, coming to this JIRA, it is undeniable that individual service folders should also be able to contribute to stack-advisor logic - it is useful when the service is not so dependent on other services and can independently provide its component-layout and configurations. Also, it is makes it easier to contribute a rich service by dropping a folder into the stack.
> >
> > With that requirement said, I think a smaller change would be to provide a mechanism by which a service can contribute logic to the stack-version's stack-advisor. This would allow for both worlds of highly-interconnected and highly-independent services to coexist in the stack-advisor framework. For an independent service, it can provide a stack-advisor script which only dealt with it's component-layout and configuration - that can be overlaid on top of the stack-version's recommendations/validations.
> >
> > My vote would be to have such an optional service-level stack-advisor script, which can be provided only when needed. We can discuss its design and how best to implement it.
Hi Srimanth, we had a meeting yesterday discussing this and that was the consensus. Service advisors will be able to plug into the stack advisor code. It will require some changes to the stacks/stack_advisor.py.
a) For the recommendConfigGroupsConfigurations and recommendConfigurations functions, the code for looping through services will need to check for service advisors:
for service in servicesList:
calculation = self.getServiceConfigurationRecommender(service)
if calculation is not None:
calculation(configurations, clusterSummary, services, hosts)
else:
advisor = createServiceAdvisor(service)
if advisor is not None:
advisor.recommendConfigurations(...)
b) For createComponentLayoutRecommendations, the code which processes the master and slave services will need to be moved into separate functions: getHostsForMasterComponent and getHostsForSlaveComponent. This will allow stack versions to overwrite the details of how those recommendations take place without messing up the code which processes service advisors. The createComponentLayoutRecommendations would end up looking something like:
advisor = createServiceAdvisor(service)
if advisor is not None:
advisor.getHostsForMasterComponent(...)
else
self.getHostsForMasterComponent(...)
c) Change def getComponentLayoutValidations from a no-op to the following:
def getComponentLayoutValidations(self, services, hosts):
servicesList = [service["StackServices"]["service_name"] for service in services["services"]]
for service in servicesList:
advisor = createServiceAdvisor(service)
if advisor is not None:
advisor.getComponentLayoutValidations(...)
This also requires the HDP 2.0.6 stack advisor to call super.getComponentLayoutValidations
d) Change def getConfigurationsValidationItems from a no-op to the following:
def getConfigurationsValidationItems(self, services, hosts):
servicesList = [service["StackServices"]["service_name"] for service in services["services"]]
for service in servicesList:
advisor = createServiceAdvisor(service)
if advisor is not None:
advisor.getConfigurationsValidationItems(...)
This also requires the HDP 2.0.6 stack advisor to call super.getConfigurationsValidationItems
I'm sure there are also a number of lesser functions which would need to handle things differently.
One thing I am still struggling with is whether I should create a default service advisor class. The problem with this class is that it would mostly duplicate a bunch of functions which are defined in the default stack advisor class. There are basically two options:
1) Create the default service advisor class and live with the duplication (and the need to make changes in multiple places)
2) Inherit the custom service advisor classes from the default stack advisor and live with the fact that a service is inheriting from the stack.
(See one of the comments above by Jun Aoki)
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review127197
-----------------------------------------------------------
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Srimanth Gunturi <sr...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review127197
-----------------------------------------------------------
Tim, thanks for contributing to the stack-advisor feature.
When we started the stack-advisor feature, there was a lot of interconnect between base service component-layout and configurations (HDFS, YARN, MapReduce, Hive, HBase) where common calculations were used by multiple services. To maximise readability and reuse of code, we took the 1 stack-advisor/stack-version approach. This has kept the number of scripts managable (8 scripts for all versions of HDP stack) to maintain and debug when problems occur. It is very useful for stacks where a lot of interconnected services exist and stack-advisor should account for all the usecases.
Now, coming to this JIRA, it is undeniable that individual service folders should also be able to contribute to stack-advisor logic - it is useful when the service is not so dependent on other services and can independently provide its component-layout and configurations. Also, it is makes it easier to contribute a rich service by dropping a folder into the stack.
With that requirement said, I think a smaller change would be to provide a mechanism by which a service can contribute logic to the stack-version's stack-advisor. This would allow for both worlds of highly-interconnected and highly-independent services to coexist in the stack-advisor framework. For an independent service, it can provide a stack-advisor script which only dealt with it's component-layout and configuration - that can be overlaid on top of the stack-version's recommendations/validations.
My vote would be to have such an optional service-level stack-advisor script, which can be provided only when needed. We can discuss its design and how best to implement it.
- Srimanth Gunturi
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On March 2, 2016, 6:58 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/default_stack_advisor.py, line 35
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275579#file1275579line35>
> >
> > Please include Srimanth Gunturi in the code review, thanks!
>
> Jayush Luniya wrote:
> +1 on adding Srimanth to the CR.
Could you please recommend who could be added for HAWQ and PXF to review those specific changes?
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121687
-----------------------------------------------------------
On April 12, 2016, 5:33 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated April 12, 2016, 5:33 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:945
> Total errors:0
> Total failures:0
>
> Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure their service advisors were called.
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Jayush Luniya <jl...@hortonworks.com>.
> On March 2, 2016, 6:58 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/default_stack_advisor.py, line 35
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275579#file1275579line35>
> >
> > Please include Srimanth Gunturi in the code review, thanks!
+1 on adding Srimanth to the CR.
- Jayush
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121687
-----------------------------------------------------------
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121687
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/default_stack_advisor.py (line 35)
<https://reviews.apache.org/r/44210/#comment183451>
Please include Srimanth Gunturi in the code review, thanks!
- Alejandro Fernandez
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On March 2, 2016, 4:56 a.m., Matt wrote:
> > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py, line 1
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275503#file1275503line1>
> >
> > If I were to add HAWQ 2.0.0's metainfo.xml under HDP 2.3 stack, and say that it extends from common-services, will this service_advisor be honored?
I had some difficulty with HAWQ and PXF because they are not actually in the stack and can be added by installing a separate RPM. In order for HAWQ and PXF to inherit the service_advisor.py files from common-services, they would need to implement mostly empty service advisors under the <stack>/<version>/services/<service> folder that explicitly inherit from the ones in common-services.
> On March 2, 2016, 4:56 a.m., Matt wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py, line 1
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275515#file1275515line1>
> >
> > If the service_advisor is under HDP 2.0.6's GANGLIA service folder, it is not required to repeat the stack name, the version and the service on the filename, for every combination of stack, version and service.
> >
> > This would be good enough:
> > HDP/2.0.6/services/GANGLIA/service_advisor.py
I tried having the name as you stated but had a problem with python. For example in 2.3/services/HDFS, I have the following:
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/HDFS/')
sys.path.append(PARENT_DIR)
from service_advisor_HDP22HDFS import *
If the file name is the same in 2.3 and 2.2, then "from service_advisor import *" could reference the 2.2 or the 2.3 file. I tried it with import HDP22HDFSServiceAdvisor as well and both of them failed to find the correct parent class.
The other way I could have solved this is to load all the service advisor files from the default_stack_advisor.py walking my way up the stack. This would have allowed you to avoid the inheritance code in the service_advisor.py files but it would restrict you from inheriting from outside of your stack (unless at that point you started including the inheritance code and the change of file names).
> On March 2, 2016, 4:56 a.m., Matt wrote:
> > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py, line 22
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275586#file1275586line22>
> >
> > If all the stack_advisor.py files are being refactored into service level advisors, the unit tests should be at service level as well.
> >
> > This would bring better modularity for introducing a new service:
> > - Add a new service to the stack
> > - Implement the service advisor for that service
> > - Implement unit test for the service advisor for that service (For the current patch, the tests are still with stack advisor and not service advisor)
I agree but hopefully that can be a future enhancement? I'd rather get this included and then work on refactoring the test code.
> On March 2, 2016, 4:56 a.m., Matt wrote:
> > ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py, line 1
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275592#file1275592line1>
> >
> > Why are the old stack_advisors retained under test folder?
They are retained because there was some test code that wanted to run a test against the old stack advisor and the new one. It is a performance test, I guess making sure the new stack advisor doesn't slow things down.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121576
-----------------------------------------------------------
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Tim Thorpe <tt...@ca.ibm.com>.
> On March 2, 2016, 4:56 a.m., Matt wrote:
> > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py, line 1
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275503#file1275503line1>
> >
> > If I were to add HAWQ 2.0.0's metainfo.xml under HDP 2.3 stack, and say that it extends from common-services, will this service_advisor be honored?
>
> Tim Thorpe wrote:
> I had some difficulty with HAWQ and PXF because they are not actually in the stack and can be added by installing a separate RPM. In order for HAWQ and PXF to inherit the service_advisor.py files from common-services, they would need to implement mostly empty service advisors under the <stack>/<version>/services/<service> folder that explicitly inherit from the ones in common-services.
>
> Matt wrote:
> Is there a way we can avoid having this empty service_advisor?
>
> service X in common-services has a service_advisor
> service X in HDP 2.2 has no service_advisor - should inherit from common-services
> service X in HDP 2.3 has service_advisor - should inherit from common-services + add new recommendations/validations from HDP 2.3 X's service_advisor
>
> Something similar to have themes can be managed or how metainfo.xml has extends property for stack definiton
Hi Matt, I can add the service advisor in the code that reads the stack/services from the file system. This would allow me to know which service_advisor.py file should be used (either because it is inherited or because it is defined in the current stack version). This information would then need to be passed to the stack advisor in the services.json. I don't think the service advisor should handle implicite inheritance between stack versions by that I mean in that your example:
service X in common-services has a service_advisor
service X in HDP 2.2 has no service_advisor - should inherit from common-services
service X in HDP 2.3 has service_advisor - only inherits from common-services if the service_advisor.py adds that logic
Basically this means that service X in HDP 2.3 would have 2 options to replace the service_advisor.py in common-services or inherit from it and add new recommendations/validations.
> On March 2, 2016, 4:56 a.m., Matt wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py, line 1
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275515#file1275515line1>
> >
> > If the service_advisor is under HDP 2.0.6's GANGLIA service folder, it is not required to repeat the stack name, the version and the service on the filename, for every combination of stack, version and service.
> >
> > This would be good enough:
> > HDP/2.0.6/services/GANGLIA/service_advisor.py
>
> Tim Thorpe wrote:
> I tried having the name as you stated but had a problem with python. For example in 2.3/services/HDFS, I have the following:
>
> SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
> PARENT_DIR = os.path.join(SCRIPT_DIR, '../../../2.2/services/HDFS/')
> sys.path.append(PARENT_DIR)
> from service_advisor_HDP22HDFS import *
>
> If the file name is the same in 2.3 and 2.2, then "from service_advisor import *" could reference the 2.2 or the 2.3 file. I tried it with import HDP22HDFSServiceAdvisor as well and both of them failed to find the correct parent class.
>
> The other way I could have solved this is to load all the service advisor files from the default_stack_advisor.py walking my way up the stack. This would have allowed you to avoid the inheritance code in the service_advisor.py files but it would restrict you from inheriting from outside of your stack (unless at that point you started including the inheritance code and the change of file names).
I have figured out a solution for this. So the naming of the service advisor file would always be service_advisor.py without being prefaced with the stack name, version and service name.
> On March 2, 2016, 4:56 a.m., Matt wrote:
> > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py, line 22
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275586#file1275586line22>
> >
> > If all the stack_advisor.py files are being refactored into service level advisors, the unit tests should be at service level as well.
> >
> > This would bring better modularity for introducing a new service:
> > - Add a new service to the stack
> > - Implement the service advisor for that service
> > - Implement unit test for the service advisor for that service (For the current patch, the tests are still with stack advisor and not service advisor)
>
> Tim Thorpe wrote:
> I agree but hopefully that can be a future enhancement? I'd rather get this included and then work on refactoring the test code.
The stack_advisor.py is no longer going to be refactored into service level advisors. The change will only allow for custom services (or later on for core services) to be written with service advisors that plug into the stack advisor framework. If and when core services do get refactored into service level advisors, the unit tests for those services can be refactored at that time.
- Tim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121576
-----------------------------------------------------------
On March 1, 2016, 4:53 p.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 4:53 p.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Matt <mi...@gmail.com>.
> On March 1, 2016, 8:56 p.m., Matt wrote:
> > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py, line 1
> > <https://reviews.apache.org/r/44210/diff/1/?file=1275503#file1275503line1>
> >
> > If I were to add HAWQ 2.0.0's metainfo.xml under HDP 2.3 stack, and say that it extends from common-services, will this service_advisor be honored?
>
> Tim Thorpe wrote:
> I had some difficulty with HAWQ and PXF because they are not actually in the stack and can be added by installing a separate RPM. In order for HAWQ and PXF to inherit the service_advisor.py files from common-services, they would need to implement mostly empty service advisors under the <stack>/<version>/services/<service> folder that explicitly inherit from the ones in common-services.
Is there a way we can avoid having this empty service_advisor?
service X in common-services has a service_advisor
service X in HDP 2.2 has no service_advisor - should inherit from common-services
service X in HDP 2.3 has service_advisor - should inherit from common-services + add new recommendations/validations from HDP 2.3 X's service_advisor
Something similar to have themes can be managed or how metainfo.xml has extends property for stack definiton
- Matt
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121576
-----------------------------------------------------------
On March 1, 2016, 8:53 a.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 8:53 a.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be
pushed down to the services
Posted by Matt <mi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review121576
-----------------------------------------------------------
Would have been better to do this on an incremental basis (starting with one service: implement + test) and not a single big patch, for the ease of testing and handling the patch.
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py (line 1)
<https://reviews.apache.org/r/44210/#comment183313>
If I were to add HAWQ 2.0.0's metainfo.xml under HDP 2.3 stack, and say that it extends from common-services, will this service_advisor be honored?
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py (line 1)
<https://reviews.apache.org/r/44210/#comment183314>
If the service_advisor is under HDP 2.0.6's GANGLIA service folder, it is not required to repeat the stack name, the version and the service on the filename, for every combination of stack, version and service.
This would be good enough:
HDP/2.0.6/services/GANGLIA/service_advisor.py
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py (line 22)
<https://reviews.apache.org/r/44210/#comment183316>
If all the stack_advisor.py files are being refactored into service level advisors, the unit tests should be at service level as well.
This would bring better modularity for introducing a new service:
- Add a new service to the stack
- Implement the service advisor for that service
- Implement unit test for the service advisor for that service (For the current patch, the tests are still with stack advisor and not service advisor)
ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py (line 1)
<https://reviews.apache.org/r/44210/#comment183315>
Why are the old stack_advisors retained under test folder?
- Matt
On March 1, 2016, 8:53 a.m., Tim Thorpe wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> -----------------------------------------------------------
>
> (Updated March 1, 2016, 8:53 a.m.)
>
>
> Review request for Ambari, Sumit Mohanty and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Currently the stack advisor is defined under each stack version such as HDP/2.3. The problem with this is that it restricts the services that can be added to the stack. If a custom service is to be added, they would need to modify the stack advisor. If the configuration recommendation and validation can be done at the service level then the custom service could just include their own recommendations and validations separately.
>
>
> Diffs
> -----
>
> ambari-agent/pom.xml c2c993f
> ambari-server/src/main/assemblies/server.xml e1a4919
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py PRE-CREATION
> ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py 53591cd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py f6f8cde
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/service_advisor_HDP21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/service_advisor_HDP21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/OOZIE/service_advisor_HDP21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/service_advisor_HDP21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/service_advisor_HDP21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/YARN/service_advisor_HDP21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py 3a552b8
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/metainfo.xml PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/AMBARI_METRICS/service_advisor_HDP22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/service_advisor_HDP22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/service_advisor_HDP22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/service_advisor_HDP22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KAFKA/service_advisor_HDP22KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/service_advisor_HDP22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/RANGER/service_advisor_HDP22RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/STORM/service_advisor_HDP22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/TEZ/service_advisor_HDP22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/service_advisor_HDP22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/stack_advisor.py bffbc26
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/service_advisor_HDP23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/service_advisor_HDP23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/service_advisor_HDP23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KAFKA/service_advisor_HDP23KAFKA.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/service_advisor_HDP23RANGER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER_KMS/service_advisor_HDP23RANGER_KMS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/service_advisor_HDP23SPARK.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/TEZ/service_advisor_HDP23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/service_advisor_HDP23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.4/services/stack_advisor.py 985c101
> ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 9dd5cc7
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py af269de
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/service_advisor_HDPWIN21AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/FALCON/service_advisor_HDPWIN21FALCON.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HBASE/service_advisor_HDPWIN21HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/service_advisor_HDPWIN21HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HIVE/service_advisor_HDPWIN21HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/OOZIE/service_advisor_HDPWIN21OOZIE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/STORM/service_advisor_HDPWIN21STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/TEZ/service_advisor_HDPWIN21TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/YARN/service_advisor_HDPWIN21YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/ZOOKEEPER/service_advisor_HDPWIN21ZOOKEEPER.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/stack_advisor.py b99c484
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/AMBARI_METRICS/service_advisor_HDPWIN22AMBARI_METRICS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HBASE/service_advisor_HDPWIN22HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HDFS/service_advisor_HDPWIN22HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/HIVE/service_advisor_HDPWIN22HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/KNOX/service_advisor_HDPWIN22KNOX.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/STORM/service_advisor_HDPWIN22STORM.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/TEZ/service_advisor_HDPWIN22TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.2/services/YARN/service_advisor_HDPWIN22YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HBASE/service_advisor_HDPWIN23HBASE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HDFS/service_advisor_HDPWIN23HDFS.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/HIVE/service_advisor_HDPWIN23HIVE.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/TEZ/service_advisor_HDPWIN23TEZ.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/YARN/service_advisor_HDPWIN23YARN.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/stack_advisor.py 3631c88
> ambari-server/src/main/resources/stacks/default_stack_advisor.py PRE-CREATION
> ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25
> ambari-server/src/test/python/TestStackAdvisor.py 149ae1d
> ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py e15582e
> ambari-server/src/test/python/stacks/2.1/common/test_stack_advisor.py 7ede5a3
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py d95e6c7
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor_perf.py 5b8c30c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8932bde
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/YARN/service_advisor_XYZ100YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.0/services/stack_advisor.py ba140bb
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/YARN/service_advisor_XYZ101YARN.py PRE-CREATION
> ambari-server/src/test/resources/stacks/XYZ/1.0.1/services/stack_advisor.py 74a4b31
> ambari-server/src/test/resources/stacks/old/HDP206stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP21stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP22stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP23stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP24stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP25stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/HDP26stack_advisor.py PRE-CREATION
> ambari-server/src/test/resources/stacks/old/stack_advisor.py PRE-CREATION
>
> Diff: https://reviews.apache.org/r/44210/diff/
>
>
> Testing
> -------
>
> Ran all the non java unit tests.
>
> Total run:921
> Total errors:1
> Total failures:0
>
> The only test which failed also failed on a fresh trunk test run: test_start (test_metrics_grafana.TestMetricsGrafana)
>
>
> Thanks,
>
> Tim Thorpe
>
>