You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2015/01/09 19:08:44 UTC
Review Request 29769: Upgrade Pack for Oozie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29769/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez and Nate Cole.
Bugs: AMBARI-9072
https://issues.apache.org/jira/browse/AMBARI-9072
Repository: ambari
Description
-------
Create the HDP upgrade pack for Oozie according to the runbook. The following steps will be performed:
- Backup and restoration of Oozie configuration files
- Backup of Oozie database
- Installation of a new sharelib in HDFS
- Preparation of the new Oozie WAR
Diffs
-----
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py a7313f3
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py a129bce
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py PRE-CREATION
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py 59966fd
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py 57ea5c1
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml ed6b50c
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4324040
ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json PRE-CREATION
Diff: https://reviews.apache.org/r/29769/diff/
Testing
-------
OK
----------------------------------------------------------------------
Total run:572
Total errors:0
Total failures:0
OK
Thanks,
Jonathan Hurley
Re: Review Request 29769: Upgrade Pack for Oozie
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29769/#review67495
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On Jan. 9, 2015, 6:53 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29769/
> -----------------------------------------------------------
>
> (Updated Jan. 9, 2015, 6:53 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-9072
> https://issues.apache.org/jira/browse/AMBARI-9072
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Create the HDP upgrade pack for Oozie according to the runbook. The following steps will be performed:
>
> - Backup and restoration of Oozie configuration files
> - Backup of Oozie database
> - Installation of a new sharelib in HDFS
> - Preparation of the new Oozie WAR
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py a7313f3
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py a129bce
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py PRE-CREATION
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py 59966fd
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py 57ea5c1
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml ed6b50c
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4324040
> ambari-server/src/test/python/stacks/2.2/configs/falcon-upgrade.json dae143a
> ambari-server/src/test/python/stacks/2.2/configs/hive-upgrade.json 876e200
> ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/29769/diff/
>
>
> Testing
> -------
>
> OK
> ----------------------------------------------------------------------
> Total run:572
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 29769: Upgrade Pack for Oozie
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29769/#review67494
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On Jan. 9, 2015, 1:53 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29769/
> -----------------------------------------------------------
>
> (Updated Jan. 9, 2015, 1:53 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-9072
> https://issues.apache.org/jira/browse/AMBARI-9072
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Create the HDP upgrade pack for Oozie according to the runbook. The following steps will be performed:
>
> - Backup and restoration of Oozie configuration files
> - Backup of Oozie database
> - Installation of a new sharelib in HDFS
> - Preparation of the new Oozie WAR
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py a7313f3
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py a129bce
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py PRE-CREATION
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py 59966fd
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py 57ea5c1
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml ed6b50c
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4324040
> ambari-server/src/test/python/stacks/2.2/configs/falcon-upgrade.json dae143a
> ambari-server/src/test/python/stacks/2.2/configs/hive-upgrade.json 876e200
> ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/29769/diff/
>
>
> Testing
> -------
>
> OK
> ----------------------------------------------------------------------
> Total run:572
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 29769: Upgrade Pack for Oozie
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29769/
-----------------------------------------------------------
(Updated Jan. 9, 2015, 1:53 p.m.)
Review request for Ambari, Alejandro Fernandez and Nate Cole.
Bugs: AMBARI-9072
https://issues.apache.org/jira/browse/AMBARI-9072
Repository: ambari
Description
-------
Create the HDP upgrade pack for Oozie according to the runbook. The following steps will be performed:
- Backup and restoration of Oozie configuration files
- Backup of Oozie database
- Installation of a new sharelib in HDFS
- Preparation of the new Oozie WAR
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py a7313f3
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py a129bce
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py PRE-CREATION
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py 59966fd
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py 57ea5c1
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml ed6b50c
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4324040
ambari-server/src/test/python/stacks/2.2/configs/falcon-upgrade.json dae143a
ambari-server/src/test/python/stacks/2.2/configs/hive-upgrade.json 876e200
ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json PRE-CREATION
Diff: https://reviews.apache.org/r/29769/diff/
Testing
-------
OK
----------------------------------------------------------------------
Total run:572
Total errors:0
Total failures:0
OK
Thanks,
Jonathan Hurley
Re: Review Request 29769: Upgrade Pack for Oozie
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Jan. 9, 2015, 1:26 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py, line 57
> > <https://reviews.apache.org/r/29769/diff/1/?file=814721#file814721line57>
> >
> > In the future, other stacks may have their own logic for how to do pre_rolling_restart, so I suggest for consistency keeping the hdp-select logic inside the "if" block that checks the hdp stack version.
In general, I dislike functions that have a single if-statement and the entire logic in the if. When a new stack is added, this function can be changed then.
> On Jan. 9, 2015, 1:26 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py, line 23
> > <https://reviews.apache.org/r/29769/diff/1/?file=814725#file814725line23>
> >
> > Thank you for switching to named imports
It prevents developer confusion, pycharm confusion, and Jonathan's frustration :)
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29769/#review67470
-----------------------------------------------------------
On Jan. 9, 2015, 1:08 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29769/
> -----------------------------------------------------------
>
> (Updated Jan. 9, 2015, 1:08 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-9072
> https://issues.apache.org/jira/browse/AMBARI-9072
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Create the HDP upgrade pack for Oozie according to the runbook. The following steps will be performed:
>
> - Backup and restoration of Oozie configuration files
> - Backup of Oozie database
> - Installation of a new sharelib in HDFS
> - Preparation of the new Oozie WAR
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py a7313f3
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py a129bce
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py PRE-CREATION
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py 59966fd
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py 57ea5c1
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml ed6b50c
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4324040
> ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/29769/diff/
>
>
> Testing
> -------
>
> OK
> ----------------------------------------------------------------------
> Total run:572
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 29769: Upgrade Pack for Oozie
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29769/#review67470
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py
<https://reviews.apache.org/r/29769/#comment111501>
In the future, other stacks may have their own logic for how to do pre_rolling_restart, so I suggest for consistency keeping the hdp-select logic inside the "if" block that checks the hdp stack version.
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py
<https://reviews.apache.org/r/29769/#comment111503>
Similar comment as above.
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py
<https://reviews.apache.org/r/29769/#comment111504>
Thank you for switching to named imports
- Alejandro Fernandez
On Jan. 9, 2015, 6:08 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29769/
> -----------------------------------------------------------
>
> (Updated Jan. 9, 2015, 6:08 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-9072
> https://issues.apache.org/jira/browse/AMBARI-9072
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Create the HDP upgrade pack for Oozie according to the runbook. The following steps will be performed:
>
> - Backup and restoration of Oozie configuration files
> - Backup of Oozie database
> - Installation of a new sharelib in HDFS
> - Preparation of the new Oozie WAR
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py a7313f3
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py a129bce
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py PRE-CREATION
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py 59966fd
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py 57ea5c1
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml ed6b50c
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4324040
> ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/29769/diff/
>
>
> Testing
> -------
>
> OK
> ----------------------------------------------------------------------
> Total run:572
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 29769: Upgrade Pack for Oozie
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Jan. 9, 2015, 1:18 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py, lines 71-72
> > <https://reviews.apache.org/r/29769/diff/1/?file=814723#file814723line71>
> >
> > Where in the world did this come from?! I hope it's doc'ed somewhere. On my deployment /usr/bin/oozie is symlinked to /usr/hdp/current/oozie-client/bin/oozie
Oozie is not documented at all :)
This is a valid step, however, and I have confirmed that the link is recreated after the upgrade is done.
> On Jan. 9, 2015, 1:18 p.m., Nate Cole wrote:
> > ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json, lines 1-4
> > <https://reviews.apache.org/r/29769/diff/1/?file=814728#file814728line1>
> >
> > Nit: I usually remove all the configs that aren't the one I'm testing. It takes the file from a thousand to about 150 lines.
Good point - more toiling in test land for me. I will update the patch.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29769/#review67464
-----------------------------------------------------------
On Jan. 9, 2015, 1:08 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29769/
> -----------------------------------------------------------
>
> (Updated Jan. 9, 2015, 1:08 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-9072
> https://issues.apache.org/jira/browse/AMBARI-9072
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Create the HDP upgrade pack for Oozie according to the runbook. The following steps will be performed:
>
> - Backup and restoration of Oozie configuration files
> - Backup of Oozie database
> - Installation of a new sharelib in HDFS
> - Preparation of the new Oozie WAR
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py a7313f3
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py a129bce
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py PRE-CREATION
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py 59966fd
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py 57ea5c1
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml ed6b50c
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4324040
> ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/29769/diff/
>
>
> Testing
> -------
>
> OK
> ----------------------------------------------------------------------
> Total run:572
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 29769: Upgrade Pack for Oozie
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29769/#review67464
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py
<https://reviews.apache.org/r/29769/#comment111495>
Where in the world did this come from?! I hope it's doc'ed somewhere. On my deployment /usr/bin/oozie is symlinked to /usr/hdp/current/oozie-client/bin/oozie
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml
<https://reviews.apache.org/r/29769/#comment111492>
Oozie should be marked skippable
ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json
<https://reviews.apache.org/r/29769/#comment111493>
Nit: I usually remove all the configs that aren't the one I'm testing. It takes the file from a thousand to about 150 lines.
- Nate Cole
On Jan. 9, 2015, 1:08 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29769/
> -----------------------------------------------------------
>
> (Updated Jan. 9, 2015, 1:08 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-9072
> https://issues.apache.org/jira/browse/AMBARI-9072
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Create the HDP upgrade pack for Oozie according to the runbook. The following steps will be performed:
>
> - Backup and restoration of Oozie configuration files
> - Backup of Oozie database
> - Installation of a new sharelib in HDFS
> - Preparation of the new Oozie WAR
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py a7313f3
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py a129bce
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py PRE-CREATION
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py 59966fd
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py 57ea5c1
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml ed6b50c
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 4324040
> ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/29769/diff/
>
>
> Testing
> -------
>
> OK
> ----------------------------------------------------------------------
> Total run:572
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>