You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2015/02/13 19:38:46 UTC
Review Request 31014: RU - Include Spark in Upgrade Pack
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31014/
-----------------------------------------------------------
Review request for Ambari, Jonathan Hurley, Nate Cole, and Tom Beerbower.
Bugs: AMBARI-9635
https://issues.apache.org/jira/browse/AMBARI-9635
Repository: ambari
Description
-------
Include Spark in upgrade pack
Diffs
-----
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 06389c7
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py 8189039
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py 2391286
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 5305b1f
Diff: https://reviews.apache.org/r/31014/diff/
Testing
-------
Tested on a 3-node cluster that did an upgrade from HDP 2.2.2* to HDP 2.2.2.*, since Spark requires HDP 2.2.1 or greater.
Installing the Spark bits did not create the folder in /usr/hdp/$version/spark, so I copied the files manually.
Thanks,
Alejandro Fernandez
Re: Review Request 31014: RU - Include Spark in Upgrade Pack
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31014/#review72434
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py
<https://reviews.apache.org/r/31014/#comment118490>
You shouldn't need that "params." there - the format function works on the top-level variables.
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py
<https://reviews.apache.org/r/31014/#comment118491>
See other comment about "params."
- Nate Cole
On Feb. 13, 2015, 2:29 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31014/
> -----------------------------------------------------------
>
> (Updated Feb. 13, 2015, 2:29 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-9635
> https://issues.apache.org/jira/browse/AMBARI-9635
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Include Spark in upgrade pack
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 06389c7
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py 8189039
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py 2391286
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 5305b1f
>
> Diff: https://reviews.apache.org/r/31014/diff/
>
>
> Testing
> -------
>
> Tested on a 3-node cluster that did an upgrade from HDP 2.2.2* to HDP 2.2.2.*, since Spark requires HDP 2.2.1 or greater.
> Installing the Spark bits did not create the folder in /usr/hdp/$version/spark, so I copied the files manually.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 31014: RU - Include Spark in Upgrade Pack
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31014/#review72718
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On Feb. 16, 2015, 8:46 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31014/
> -----------------------------------------------------------
>
> (Updated Feb. 16, 2015, 8:46 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-9635
> https://issues.apache.org/jira/browse/AMBARI-9635
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Include Spark in upgrade pack
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 9965d2f
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py ea6c67d
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py 67cbb34
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 5305b1f
>
> Diff: https://reviews.apache.org/r/31014/diff/
>
>
> Testing
> -------
>
> Tested on a 3-node cluster that did an upgrade from HDP 2.2.2* to HDP 2.2.2.*, since Spark requires HDP 2.2.1 or greater.
> Installing the Spark bits did not create the folder in /usr/hdp/$version/spark, so I copied the files manually.
>
> Python unit tests on ambari-server
>
> ----------------------------------------------------------------------
> Total run:594
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 31014: RU - Include Spark in Upgrade Pack
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31014/#review72720
-----------------------------------------------------------
Ship it!
Ship It!
- Tom Beerbower
On Feb. 17, 2015, 1:46 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31014/
> -----------------------------------------------------------
>
> (Updated Feb. 17, 2015, 1:46 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-9635
> https://issues.apache.org/jira/browse/AMBARI-9635
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Include Spark in upgrade pack
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 9965d2f
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py ea6c67d
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py 67cbb34
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 5305b1f
>
> Diff: https://reviews.apache.org/r/31014/diff/
>
>
> Testing
> -------
>
> Tested on a 3-node cluster that did an upgrade from HDP 2.2.2* to HDP 2.2.2.*, since Spark requires HDP 2.2.1 or greater.
> Installing the Spark bits did not create the folder in /usr/hdp/$version/spark, so I copied the files manually.
>
> Python unit tests on ambari-server
>
> ----------------------------------------------------------------------
> Total run:594
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 31014: RU - Include Spark in Upgrade Pack
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31014/#review72708
-----------------------------------------------------------
Ship it!
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml
<https://reviews.apache.org/r/31014/#comment118767>
If you didn't already, confirm with Rohit that Spark should go between Hive and Oozie and shouldn't go after Clients with the "lesser" services.
- Nate Cole
On Feb. 16, 2015, 8:46 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31014/
> -----------------------------------------------------------
>
> (Updated Feb. 16, 2015, 8:46 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-9635
> https://issues.apache.org/jira/browse/AMBARI-9635
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Include Spark in upgrade pack
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 9965d2f
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py ea6c67d
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py 67cbb34
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 5305b1f
>
> Diff: https://reviews.apache.org/r/31014/diff/
>
>
> Testing
> -------
>
> Tested on a 3-node cluster that did an upgrade from HDP 2.2.2* to HDP 2.2.2.*, since Spark requires HDP 2.2.1 or greater.
> Installing the Spark bits did not create the folder in /usr/hdp/$version/spark, so I copied the files manually.
>
> Python unit tests on ambari-server
>
> ----------------------------------------------------------------------
> Total run:594
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 31014: RU - Include Spark in Upgrade Pack
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31014/
-----------------------------------------------------------
(Updated Feb. 17, 2015, 1:46 a.m.)
Review request for Ambari, Jonathan Hurley, Nate Cole, and Tom Beerbower.
Bugs: AMBARI-9635
https://issues.apache.org/jira/browse/AMBARI-9635
Repository: ambari
Description
-------
Include Spark in upgrade pack
Diffs
-----
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 9965d2f
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py ea6c67d
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py 67cbb34
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 5305b1f
Diff: https://reviews.apache.org/r/31014/diff/
Testing (updated)
-------
Tested on a 3-node cluster that did an upgrade from HDP 2.2.2* to HDP 2.2.2.*, since Spark requires HDP 2.2.1 or greater.
Installing the Spark bits did not create the folder in /usr/hdp/$version/spark, so I copied the files manually.
Python unit tests on ambari-server
----------------------------------------------------------------------
Total run:594
Total errors:0
Total failures:0
OK
Thanks,
Alejandro Fernandez
Re: Review Request 31014: RU - Include Spark in Upgrade Pack
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31014/
-----------------------------------------------------------
(Updated Feb. 17, 2015, 1:43 a.m.)
Review request for Ambari, Jonathan Hurley, Nate Cole, and Tom Beerbower.
Changes
-------
Addressed Nate's comments.
Bugs: AMBARI-9635
https://issues.apache.org/jira/browse/AMBARI-9635
Repository: ambari
Description
-------
Include Spark in upgrade pack
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 9965d2f
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py ea6c67d
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py 67cbb34
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 5305b1f
Diff: https://reviews.apache.org/r/31014/diff/
Testing
-------
Tested on a 3-node cluster that did an upgrade from HDP 2.2.2* to HDP 2.2.2.*, since Spark requires HDP 2.2.1 or greater.
Installing the Spark bits did not create the folder in /usr/hdp/$version/spark, so I copied the files manually.
Thanks,
Alejandro Fernandez
Re: Review Request 31014: RU - Include Spark in Upgrade Pack
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31014/
-----------------------------------------------------------
(Updated Feb. 13, 2015, 7:29 p.m.)
Review request for Ambari, Jonathan Hurley, Nate Cole, and Tom Beerbower.
Bugs: AMBARI-9635
https://issues.apache.org/jira/browse/AMBARI-9635
Repository: ambari
Description
-------
Include Spark in upgrade pack
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 06389c7
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py 8189039
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py 2391286
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 5305b1f
Diff: https://reviews.apache.org/r/31014/diff/
Testing
-------
Tested on a 3-node cluster that did an upgrade from HDP 2.2.2* to HDP 2.2.2.*, since Spark requires HDP 2.2.1 or greater.
Installing the Spark bits did not create the folder in /usr/hdp/$version/spark, so I copied the files manually.
Thanks,
Alejandro Fernandez
Re: Review Request 31014: RU - Include Spark in Upgrade Pack
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31014/#review72421
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml
<https://reviews.apache.org/r/31014/#comment118481>
Will move this up to the SPARK group.
- Alejandro Fernandez
On Feb. 13, 2015, 6:38 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31014/
> -----------------------------------------------------------
>
> (Updated Feb. 13, 2015, 6:38 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-9635
> https://issues.apache.org/jira/browse/AMBARI-9635
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Include Spark in upgrade pack
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py 06389c7
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py 8189039
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_client.py 2391286
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml 5305b1f
>
> Diff: https://reviews.apache.org/r/31014/diff/
>
>
> Testing
> -------
>
> Tested on a 3-node cluster that did an upgrade from HDP 2.2.2* to HDP 2.2.2.*, since Spark requires HDP 2.2.1 or greater.
> Installing the Spark bits did not create the folder in /usr/hdp/$version/spark, so I copied the files manually.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>