You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Juanjo Marron <jm...@us.ibm.com> on 2016/03/25 19:25:05 UTC
Review Request 45338: AMBARI-15053: Stack Featurize YARN & MR services
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45338/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit Mohanty.
Bugs: AMBARI-15053
https://issues.apache.org/jira/browse/AMBARI-15053
Repository: ambari
Description
-------
Featurize HDP specific logic from YARN & MR service
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/constants.py f766a82
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py 2f0e6bf
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 2966581
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py 53b0e53
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 9fc1e32
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py fd14d0f
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 52f42b0
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py e51ca8a
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py 83bf460
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 4110d39
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py d300279
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json 97bd19c
Diff: https://reviews.apache.org/r/45338/diff/
Testing
-------
mvn clean test -DskipSurefireTests
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [17.789s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.379s]
[INFO] Ambari Web ........................................ SUCCESS [1:28.925s]
[INFO] Ambari Views ...................................... SUCCESS [7.096s]
[INFO] Ambari Admin View ................................. SUCCESS [18.829s]
[INFO] ambari-metrics .................................... SUCCESS [0.358s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.222s]
[INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [2.142s]
[INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.640s]
[INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.800s]
[INFO] Ambari Metrics Storm Sink ......................... SUCCESS [2.858s]
[INFO] Ambari Metrics Collector .......................... SUCCESS [30.150s]
[INFO] Ambari Metrics Monitor ............................ SUCCESS [3.053s]
[INFO] Ambari Metrics Grafana ............................ SUCCESS [20.581s]
[INFO] Ambari Metrics Assembly ........................... SUCCESS [39.513s]
[INFO] Ambari Server ..................................... SUCCESS [1:42.265s]
[INFO] Ambari Functional Tests ........................... SUCCESS [2.830s]
[INFO] Ambari Agent ...................................... SUCCESS [9.914s]
[INFO] Ambari Client ..................................... SUCCESS [0.286s]
[INFO] Ambari Python Client .............................. SUCCESS [1.151s]
[INFO] Ambari Groovy Client .............................. SUCCESS [17.175s]
[INFO] Ambari Shell ...................................... SUCCESS [0.251s]
[INFO] Ambari Python Shell ............................... SUCCESS [0.157s]
[INFO] Ambari Groovy Shell ............................... SUCCESS [10.952s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6:20.634s
[INFO] Finished at: Thu Mar 25 11:22:15 PDT 2016
[INFO] Final Memory: 112M/531M
Thanks,
Juanjo Marron
Re: Review Request 45338: AMBARI-15053: Stack Featurize YARN and MR
services
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45338/#review125908
-----------------------------------------------------------
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py (line 73)
<https://reviews.apache.org/r/45338/#comment188812>
We are copying Tez, Mapreduce and Slider tarballs under this condition.
- Jayush Luniya
On March 28, 2016, 4:09 p.m., Juanjo Marron wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45338/
> -----------------------------------------------------------
>
> (Updated March 28, 2016, 4:09 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit Mohanty.
>
>
> Bugs: AMBARI-15053
> https://issues.apache.org/jira/browse/AMBARI-15053
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Featurize HDP specific logic from YARN and MR service
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py f766a82
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py 2f0e6bf
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 2966581
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py 53b0e53
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 9fc1e32
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py fd14d0f
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 52f42b0
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py e51ca8a
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py 83bf460
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 4110d39
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py d300279
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json 97bd19c
>
> Diff: https://reviews.apache.org/r/45338/diff/
>
>
> Testing
> -------
>
> mvn clean test -DskipSurefireTests
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [17.789s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.379s]
> [INFO] Ambari Web ........................................ SUCCESS [1:28.925s]
> [INFO] Ambari Views ...................................... SUCCESS [7.096s]
> [INFO] Ambari Admin View ................................. SUCCESS [18.829s]
> [INFO] ambari-metrics .................................... SUCCESS [0.358s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.222s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [2.142s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.640s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.800s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [2.858s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [30.150s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [3.053s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [20.581s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [39.513s]
> [INFO] Ambari Server ..................................... SUCCESS [1:42.265s]
> [INFO] Ambari Functional Tests ........................... SUCCESS [2.830s]
> [INFO] Ambari Agent ...................................... SUCCESS [9.914s]
> [INFO] Ambari Client ..................................... SUCCESS [0.286s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.151s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [17.175s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.251s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.157s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [10.952s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 6:20.634s
> [INFO] Finished at: Thu Mar 25 11:22:15 PDT 2016
> [INFO] Final Memory: 112M/531M
>
>
> Thanks,
>
> Juanjo Marron
>
>
Re: Review Request 45338: AMBARI-15053: Stack Featurize YARN and MR
services
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45338/#review126012
-----------------------------------------------------------
Ship it!
Ship It!
- Jayush Luniya
On March 29, 2016, 8:29 p.m., Juanjo Marron wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45338/
> -----------------------------------------------------------
>
> (Updated March 29, 2016, 8:29 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit Mohanty.
>
>
> Bugs: AMBARI-15053
> https://issues.apache.org/jira/browse/AMBARI-15053
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Featurize HDP specific logic from YARN and MR service
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py f766a82
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py 2f0e6bf
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 2966581
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py 53b0e53
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 9fc1e32
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py fd14d0f
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 1a060de
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py e51ca8a
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py 83bf460
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 4110d39
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py d300279
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json 97bd19c
>
> Diff: https://reviews.apache.org/r/45338/diff/
>
>
> Testing
> -------
>
> mvn clean test -DskipSurefireTests
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [17.789s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.379s]
> [INFO] Ambari Web ........................................ SUCCESS [1:28.925s]
> [INFO] Ambari Views ...................................... SUCCESS [7.096s]
> [INFO] Ambari Admin View ................................. SUCCESS [18.829s]
> [INFO] ambari-metrics .................................... SUCCESS [0.358s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.222s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [2.142s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.640s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.800s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [2.858s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [30.150s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [3.053s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [20.581s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [39.513s]
> [INFO] Ambari Server ..................................... SUCCESS [1:42.265s]
> [INFO] Ambari Functional Tests ........................... SUCCESS [2.830s]
> [INFO] Ambari Agent ...................................... SUCCESS [9.914s]
> [INFO] Ambari Client ..................................... SUCCESS [0.286s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.151s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [17.175s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.251s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.157s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [10.952s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 6:20.634s
> [INFO] Finished at: Thu Mar 25 11:22:15 PDT 2016
> [INFO] Final Memory: 112M/531M
>
>
> Thanks,
>
> Juanjo Marron
>
>
Re: Review Request 45338: AMBARI-15053: Stack Featurize YARN and MR
services
Posted by Juanjo Marron <jm...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45338/
-----------------------------------------------------------
(Updated March 29, 2016, 8:29 p.m.)
Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit Mohanty.
Bugs: AMBARI-15053
https://issues.apache.org/jira/browse/AMBARI-15053
Repository: ambari
Description
-------
Featurize HDP specific logic from YARN and MR service
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/constants.py f766a82
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py 2f0e6bf
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 2966581
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py 53b0e53
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 9fc1e32
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py fd14d0f
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 1a060de
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py e51ca8a
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py 83bf460
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 4110d39
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py d300279
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json 97bd19c
Diff: https://reviews.apache.org/r/45338/diff/
Testing
-------
mvn clean test -DskipSurefireTests
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [17.789s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.379s]
[INFO] Ambari Web ........................................ SUCCESS [1:28.925s]
[INFO] Ambari Views ...................................... SUCCESS [7.096s]
[INFO] Ambari Admin View ................................. SUCCESS [18.829s]
[INFO] ambari-metrics .................................... SUCCESS [0.358s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.222s]
[INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [2.142s]
[INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.640s]
[INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.800s]
[INFO] Ambari Metrics Storm Sink ......................... SUCCESS [2.858s]
[INFO] Ambari Metrics Collector .......................... SUCCESS [30.150s]
[INFO] Ambari Metrics Monitor ............................ SUCCESS [3.053s]
[INFO] Ambari Metrics Grafana ............................ SUCCESS [20.581s]
[INFO] Ambari Metrics Assembly ........................... SUCCESS [39.513s]
[INFO] Ambari Server ..................................... SUCCESS [1:42.265s]
[INFO] Ambari Functional Tests ........................... SUCCESS [2.830s]
[INFO] Ambari Agent ...................................... SUCCESS [9.914s]
[INFO] Ambari Client ..................................... SUCCESS [0.286s]
[INFO] Ambari Python Client .............................. SUCCESS [1.151s]
[INFO] Ambari Groovy Client .............................. SUCCESS [17.175s]
[INFO] Ambari Shell ...................................... SUCCESS [0.251s]
[INFO] Ambari Python Shell ............................... SUCCESS [0.157s]
[INFO] Ambari Groovy Shell ............................... SUCCESS [10.952s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6:20.634s
[INFO] Finished at: Thu Mar 25 11:22:15 PDT 2016
[INFO] Final Memory: 112M/531M
Thanks,
Juanjo Marron
Re: Review Request 45338: AMBARI-15053: Stack Featurize YARN and MR
services
Posted by Juanjo Marron <jm...@us.ibm.com>.
> On March 28, 2016, 6:28 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py, line 22
> > <https://reviews.apache.org/r/45338/diff/1/?file=1315009#file1315009line22>
> >
> > Can we remove the import *?
>
> Juanjo Marron wrote:
> Most of the service scripts still include the from resource_management import * line.
> I found 123 matches searching in common-services.
> It would be great to clean up the extra/non use imports, but if @afernandez agrees, I think it is out of the scope of this stack featuraization work
I tried to remove the from resource_management import * line and other specific imports are missed, such as Script ot Fail.
It would require additional validation.
Ill create a new JIRA for imports clean up ate common-services level
- Juanjo
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45338/#review125695
-----------------------------------------------------------
On March 28, 2016, 4:09 p.m., Juanjo Marron wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45338/
> -----------------------------------------------------------
>
> (Updated March 28, 2016, 4:09 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit Mohanty.
>
>
> Bugs: AMBARI-15053
> https://issues.apache.org/jira/browse/AMBARI-15053
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Featurize HDP specific logic from YARN and MR service
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py f766a82
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py 2f0e6bf
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 2966581
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py 53b0e53
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 9fc1e32
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py fd14d0f
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 52f42b0
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py e51ca8a
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py 83bf460
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 4110d39
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py d300279
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json 97bd19c
>
> Diff: https://reviews.apache.org/r/45338/diff/
>
>
> Testing
> -------
>
> mvn clean test -DskipSurefireTests
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [17.789s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.379s]
> [INFO] Ambari Web ........................................ SUCCESS [1:28.925s]
> [INFO] Ambari Views ...................................... SUCCESS [7.096s]
> [INFO] Ambari Admin View ................................. SUCCESS [18.829s]
> [INFO] ambari-metrics .................................... SUCCESS [0.358s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.222s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [2.142s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.640s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.800s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [2.858s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [30.150s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [3.053s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [20.581s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [39.513s]
> [INFO] Ambari Server ..................................... SUCCESS [1:42.265s]
> [INFO] Ambari Functional Tests ........................... SUCCESS [2.830s]
> [INFO] Ambari Agent ...................................... SUCCESS [9.914s]
> [INFO] Ambari Client ..................................... SUCCESS [0.286s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.151s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [17.175s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.251s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.157s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [10.952s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 6:20.634s
> [INFO] Finished at: Thu Mar 25 11:22:15 PDT 2016
> [INFO] Final Memory: 112M/531M
>
>
> Thanks,
>
> Juanjo Marron
>
>
Re: Review Request 45338: AMBARI-15053: Stack Featurize YARN and MR
services
Posted by Juanjo Marron <jm...@us.ibm.com>.
> On March 28, 2016, 6:28 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py, line 56
> > <https://reviews.apache.org/r/45338/diff/1/?file=1315011#file1315011line56>
> >
> > The point of this function was to have a dictionary with the key being each possible stack name. If we always return "hadoop-client", might as well just return it as a string no matter what.
The didct value is changing depending the component. The key is always the same ("HDP" and stack_name now).
Since several services were already pushed to trunk with this same approcah, I would propose to solve first stack featurization for all the services and create a new JIRA to modify get_stack_to_component() after that.
I think the method def get_stack_to_component(self): in script.py can be redefined to obtain the stack_name at script.py level (method def get_stack_name()) and to return just a string with the component name at service level.
> On March 28, 2016, 6:28 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py, line 22
> > <https://reviews.apache.org/r/45338/diff/1/?file=1315009#file1315009line22>
> >
> > Can we remove the import *?
Most of the service scripts still include the from resource_management import * line.
I found 123 matches searching in common-services.
It would be great to clean up the extra/non use imports, but if @afernandez agrees, I think it is out of the scope of this stack featuraization work
- Juanjo
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45338/#review125695
-----------------------------------------------------------
On March 28, 2016, 4:09 p.m., Juanjo Marron wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45338/
> -----------------------------------------------------------
>
> (Updated March 28, 2016, 4:09 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit Mohanty.
>
>
> Bugs: AMBARI-15053
> https://issues.apache.org/jira/browse/AMBARI-15053
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Featurize HDP specific logic from YARN and MR service
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py f766a82
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py 2f0e6bf
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 2966581
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py 53b0e53
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 9fc1e32
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py fd14d0f
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 52f42b0
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py e51ca8a
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py 83bf460
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 4110d39
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py d300279
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json 97bd19c
>
> Diff: https://reviews.apache.org/r/45338/diff/
>
>
> Testing
> -------
>
> mvn clean test -DskipSurefireTests
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [17.789s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.379s]
> [INFO] Ambari Web ........................................ SUCCESS [1:28.925s]
> [INFO] Ambari Views ...................................... SUCCESS [7.096s]
> [INFO] Ambari Admin View ................................. SUCCESS [18.829s]
> [INFO] ambari-metrics .................................... SUCCESS [0.358s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.222s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [2.142s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.640s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.800s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [2.858s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [30.150s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [3.053s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [20.581s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [39.513s]
> [INFO] Ambari Server ..................................... SUCCESS [1:42.265s]
> [INFO] Ambari Functional Tests ........................... SUCCESS [2.830s]
> [INFO] Ambari Agent ...................................... SUCCESS [9.914s]
> [INFO] Ambari Client ..................................... SUCCESS [0.286s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.151s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [17.175s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.251s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.157s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [10.952s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 6:20.634s
> [INFO] Finished at: Thu Mar 25 11:22:15 PDT 2016
> [INFO] Final Memory: 112M/531M
>
>
> Thanks,
>
> Juanjo Marron
>
>
Re: Review Request 45338: AMBARI-15053: Stack Featurize YARN and MR
services
Posted by Juanjo Marron <jm...@us.ibm.com>.
> On March 28, 2016, 6:28 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py, line 56
> > <https://reviews.apache.org/r/45338/diff/1/?file=1315011#file1315011line56>
> >
> > The point of this function was to have a dictionary with the key being each possible stack name. If we always return "hadoop-client", might as well just return it as a string no matter what.
>
> Juanjo Marron wrote:
> The didct value is changing depending the component. The key is always the same ("HDP" and stack_name now).
>
> Since several services were already pushed to trunk with this same approcah, I would propose to solve first stack featurization for all the services and create a new JIRA to modify get_stack_to_component() after that.
>
> I think the method def get_stack_to_component(self): in script.py can be redefined to obtain the stack_name at script.py level (method def get_stack_name()) and to return just a string with the component name at service level.
Created JIRA
https://issues.apache.org/jira/browse/AMBARI-15609
- Juanjo
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45338/#review125695
-----------------------------------------------------------
On March 29, 2016, 8:29 p.m., Juanjo Marron wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45338/
> -----------------------------------------------------------
>
> (Updated March 29, 2016, 8:29 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit Mohanty.
>
>
> Bugs: AMBARI-15053
> https://issues.apache.org/jira/browse/AMBARI-15053
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Featurize HDP specific logic from YARN and MR service
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py f766a82
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py 2f0e6bf
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 2966581
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py 53b0e53
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 9fc1e32
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py fd14d0f
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 1a060de
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py e51ca8a
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py 83bf460
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 4110d39
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py d300279
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json 97bd19c
>
> Diff: https://reviews.apache.org/r/45338/diff/
>
>
> Testing
> -------
>
> mvn clean test -DskipSurefireTests
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [17.789s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.379s]
> [INFO] Ambari Web ........................................ SUCCESS [1:28.925s]
> [INFO] Ambari Views ...................................... SUCCESS [7.096s]
> [INFO] Ambari Admin View ................................. SUCCESS [18.829s]
> [INFO] ambari-metrics .................................... SUCCESS [0.358s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.222s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [2.142s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.640s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.800s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [2.858s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [30.150s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [3.053s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [20.581s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [39.513s]
> [INFO] Ambari Server ..................................... SUCCESS [1:42.265s]
> [INFO] Ambari Functional Tests ........................... SUCCESS [2.830s]
> [INFO] Ambari Agent ...................................... SUCCESS [9.914s]
> [INFO] Ambari Client ..................................... SUCCESS [0.286s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.151s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [17.175s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.251s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.157s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [10.952s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 6:20.634s
> [INFO] Finished at: Thu Mar 25 11:22:15 PDT 2016
> [INFO] Final Memory: 112M/531M
>
>
> Thanks,
>
> Juanjo Marron
>
>
Re: Review Request 45338: AMBARI-15053: Stack Featurize YARN and MR
services
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45338/#review125695
-----------------------------------------------------------
Fix it, then Ship it!
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py (line 22)
<https://reviews.apache.org/r/45338/#comment188565>
Can we remove the import *?
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py (line 68)
<https://reviews.apache.org/r/45338/#comment188568>
Same comment as below.
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py (line 23)
<https://reviews.apache.org/r/45338/#comment188566>
Can we remove import *?
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py (line 56)
<https://reviews.apache.org/r/45338/#comment188567>
The point of this function was to have a dictionary with the key being each possible stack name. If we always return "hadoop-client", might as well just return it as a string no matter what.
- Alejandro Fernandez
On March 28, 2016, 4:09 p.m., Juanjo Marron wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45338/
> -----------------------------------------------------------
>
> (Updated March 28, 2016, 4:09 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit Mohanty.
>
>
> Bugs: AMBARI-15053
> https://issues.apache.org/jira/browse/AMBARI-15053
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Featurize HDP specific logic from YARN and MR service
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py f766a82
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py 2f0e6bf
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 2966581
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py 53b0e53
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 9fc1e32
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py fd14d0f
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 52f42b0
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py e51ca8a
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py 83bf460
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 4110d39
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py d300279
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json 97bd19c
>
> Diff: https://reviews.apache.org/r/45338/diff/
>
>
> Testing
> -------
>
> mvn clean test -DskipSurefireTests
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [17.789s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.379s]
> [INFO] Ambari Web ........................................ SUCCESS [1:28.925s]
> [INFO] Ambari Views ...................................... SUCCESS [7.096s]
> [INFO] Ambari Admin View ................................. SUCCESS [18.829s]
> [INFO] ambari-metrics .................................... SUCCESS [0.358s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.222s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [2.142s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.640s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.800s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [2.858s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [30.150s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [3.053s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [20.581s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [39.513s]
> [INFO] Ambari Server ..................................... SUCCESS [1:42.265s]
> [INFO] Ambari Functional Tests ........................... SUCCESS [2.830s]
> [INFO] Ambari Agent ...................................... SUCCESS [9.914s]
> [INFO] Ambari Client ..................................... SUCCESS [0.286s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.151s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [17.175s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.251s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.157s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [10.952s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 6:20.634s
> [INFO] Finished at: Thu Mar 25 11:22:15 PDT 2016
> [INFO] Final Memory: 112M/531M
>
>
> Thanks,
>
> Juanjo Marron
>
>
Re: Review Request 45338: AMBARI-15053: Stack Featurize YARN and MR
services
Posted by Juanjo Marron <jm...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45338/
-----------------------------------------------------------
(Updated March 28, 2016, 4:09 p.m.)
Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit Mohanty.
Summary (updated)
-----------------
AMBARI-15053: Stack Featurize YARN and MR services
Bugs: AMBARI-15053
https://issues.apache.org/jira/browse/AMBARI-15053
Repository: ambari
Description (updated)
-------
Featurize HDP specific logic from YARN and MR service
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/constants.py f766a82
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py 2f0e6bf
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py 2966581
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py 53b0e53
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py 9fc1e32
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py fd14d0f
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 52f42b0
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py e51ca8a
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py 83bf460
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 4110d39
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py d300279
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json 97bd19c
Diff: https://reviews.apache.org/r/45338/diff/
Testing (updated)
-------
mvn clean test -DskipSurefireTests
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [17.789s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.379s]
[INFO] Ambari Web ........................................ SUCCESS [1:28.925s]
[INFO] Ambari Views ...................................... SUCCESS [7.096s]
[INFO] Ambari Admin View ................................. SUCCESS [18.829s]
[INFO] ambari-metrics .................................... SUCCESS [0.358s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.222s]
[INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [2.142s]
[INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.640s]
[INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.800s]
[INFO] Ambari Metrics Storm Sink ......................... SUCCESS [2.858s]
[INFO] Ambari Metrics Collector .......................... SUCCESS [30.150s]
[INFO] Ambari Metrics Monitor ............................ SUCCESS [3.053s]
[INFO] Ambari Metrics Grafana ............................ SUCCESS [20.581s]
[INFO] Ambari Metrics Assembly ........................... SUCCESS [39.513s]
[INFO] Ambari Server ..................................... SUCCESS [1:42.265s]
[INFO] Ambari Functional Tests ........................... SUCCESS [2.830s]
[INFO] Ambari Agent ...................................... SUCCESS [9.914s]
[INFO] Ambari Client ..................................... SUCCESS [0.286s]
[INFO] Ambari Python Client .............................. SUCCESS [1.151s]
[INFO] Ambari Groovy Client .............................. SUCCESS [17.175s]
[INFO] Ambari Shell ...................................... SUCCESS [0.251s]
[INFO] Ambari Python Shell ............................... SUCCESS [0.157s]
[INFO] Ambari Groovy Shell ............................... SUCCESS [10.952s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6:20.634s
[INFO] Finished at: Thu Mar 25 11:22:15 PDT 2016
[INFO] Final Memory: 112M/531M
Thanks,
Juanjo Marron