You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2015/11/06 03:52:39 UTC
Review Request 40004: AMBARI-13755. Express Upgrade: Failed to
Downgrade Namenode
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40004/
-----------------------------------------------------------
Review request for Ambari, Dmitro Lisnichenko, Jayush Luniya, Nate Cole, and Swapan Shridhar.
Bugs: AMBARI-13755
https://issues.apache.org/jira/browse/AMBARI-13755
Repository: ambari
Description
-------
Setup:
* Express Upgrade on HDP 2.2.6.0 -> HDP 2.3.0.0
* Services: HDFS + ZK
* Upgrade completes successfully
* Downgrade after reaching Finalize Upgrade
* Downgrade blocked at Namenode restart
This is happening because the downgrade command to HDFS needs to coincide with the same command that was used to upgrade it.
HDFS supports two types of upgrades:
1. -rollingUpgrade started (allows downgrading so that there is no data loss. Downgrade is either "-rollingUpgrade started" or "-rollingUpgrade downgrade")
2. -upgrade (only allows rollback, so any data that was added since the upgrade will be lost. Rollback is done with "-rollback" command).
Because Express Upgrade will support only these paths, we will use option 1 for all cases.
HDP 2.1->2.3 (only upgrade)
HDP 2.2->2.3 (both upgrade and downgrade)
HDP 2.2->2.2.* (both upgrade and downgrade)
HDP 2.3->2.3.* (both upgrade and downgrade)
Diffs
-----
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py f941164
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py ade2ec4
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py c8c057d
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 0f96d03
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 3a518cf
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml d6328dc
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 2f807fd
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 51ccf59
Diff: https://reviews.apache.org/r/40004/diff/
Testing
-------
Tested in the following scenarios.
HDP 2.1->2.3:
Express Upgrade: setup with and without HA
HDP 2.2->2.3:
Express Upgrade: setup with and without HA, followed upgrade and then downgrade.
Rolling Upgrade: setup with HA, followed upgrade and then downgrade.
Thanks,
Alejandro Fernandez
Re: Review Request 40004: AMBARI-13755. Express Upgrade: Failed to
Downgrade Namenode
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Nov. 6, 2015, 5:40 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py, lines 200-201
> > <https://reviews.apache.org/r/40004/diff/1/?file=1117550#file1117550line200>
> >
> > Is there anything we can do instead of sleeping? Any command we can run to check?
We are already waiting for it to come out of safemode. This is just a precaution.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40004/#review105473
-----------------------------------------------------------
On Nov. 6, 2015, 2:52 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40004/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2015, 2:52 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jayush Luniya, Nate Cole, and Swapan Shridhar.
>
>
> Bugs: AMBARI-13755
> https://issues.apache.org/jira/browse/AMBARI-13755
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Setup:
>
> * Express Upgrade on HDP 2.2.6.0 -> HDP 2.3.0.0
> * Services: HDFS + ZK
> * Upgrade completes successfully
> * Downgrade after reaching Finalize Upgrade
> * Downgrade blocked at Namenode restart
>
> This is happening because the downgrade command to HDFS needs to coincide with the same command that was used to upgrade it.
> HDFS supports two types of upgrades:
> 1. -rollingUpgrade started (allows downgrading so that there is no data loss. Downgrade is either "-rollingUpgrade started" or "-rollingUpgrade downgrade")
> 2. -upgrade (only allows rollback, so any data that was added since the upgrade will be lost. Rollback is done with "-rollback" command).
>
> Because Express Upgrade will support only these paths, we will use option 1 for all cases.
> HDP 2.1->2.3 (only upgrade)
> HDP 2.2->2.3 (both upgrade and downgrade)
> HDP 2.2->2.2.* (both upgrade and downgrade)
> HDP 2.3->2.3.* (both upgrade and downgrade)
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py f941164
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py ade2ec4
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py c8c057d
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 0f96d03
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 3a518cf
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml d6328dc
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 2f807fd
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 51ccf59
>
> Diff: https://reviews.apache.org/r/40004/diff/
>
>
> Testing
> -------
>
> Tested in the following scenarios.
> HDP 2.1->2.3:
> Express Upgrade: setup with and without HA
>
> HDP 2.2->2.3:
> Express Upgrade: setup with and without HA, followed upgrade and then downgrade.
> Rolling Upgrade: setup with HA, followed upgrade and then downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 40004: AMBARI-13755. Express Upgrade: Failed to
Downgrade Namenode
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40004/#review105473
-----------------------------------------------------------
Ship it!
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py (lines 198 - 199)
<https://reviews.apache.org/r/40004/#comment164094>
Is there anything we can do instead of sleeping? Any command we can run to check?
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml (line 125)
<https://reviews.apache.org/r/40004/#comment164095>
Thank you!
- Nate Cole
On Nov. 5, 2015, 9:52 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40004/
> -----------------------------------------------------------
>
> (Updated Nov. 5, 2015, 9:52 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jayush Luniya, Nate Cole, and Swapan Shridhar.
>
>
> Bugs: AMBARI-13755
> https://issues.apache.org/jira/browse/AMBARI-13755
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Setup:
>
> * Express Upgrade on HDP 2.2.6.0 -> HDP 2.3.0.0
> * Services: HDFS + ZK
> * Upgrade completes successfully
> * Downgrade after reaching Finalize Upgrade
> * Downgrade blocked at Namenode restart
>
> This is happening because the downgrade command to HDFS needs to coincide with the same command that was used to upgrade it.
> HDFS supports two types of upgrades:
> 1. -rollingUpgrade started (allows downgrading so that there is no data loss. Downgrade is either "-rollingUpgrade started" or "-rollingUpgrade downgrade")
> 2. -upgrade (only allows rollback, so any data that was added since the upgrade will be lost. Rollback is done with "-rollback" command).
>
> Because Express Upgrade will support only these paths, we will use option 1 for all cases.
> HDP 2.1->2.3 (only upgrade)
> HDP 2.2->2.3 (both upgrade and downgrade)
> HDP 2.2->2.2.* (both upgrade and downgrade)
> HDP 2.3->2.3.* (both upgrade and downgrade)
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py f941164
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py ade2ec4
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py c8c057d
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 0f96d03
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 3a518cf
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml d6328dc
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 2f807fd
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 51ccf59
>
> Diff: https://reviews.apache.org/r/40004/diff/
>
>
> Testing
> -------
>
> Tested in the following scenarios.
> HDP 2.1->2.3:
> Express Upgrade: setup with and without HA
>
> HDP 2.2->2.3:
> Express Upgrade: setup with and without HA, followed upgrade and then downgrade.
> Rolling Upgrade: setup with HA, followed upgrade and then downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 40004: AMBARI-13755. Express Upgrade: Failed to
Downgrade Namenode
Posted by Jayush Luniya <jl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40004/#review105477
-----------------------------------------------------------
Ship it!
Ship It!
- Jayush Luniya
On Nov. 6, 2015, 2:52 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40004/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2015, 2:52 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jayush Luniya, Nate Cole, and Swapan Shridhar.
>
>
> Bugs: AMBARI-13755
> https://issues.apache.org/jira/browse/AMBARI-13755
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Setup:
>
> * Express Upgrade on HDP 2.2.6.0 -> HDP 2.3.0.0
> * Services: HDFS + ZK
> * Upgrade completes successfully
> * Downgrade after reaching Finalize Upgrade
> * Downgrade blocked at Namenode restart
>
> This is happening because the downgrade command to HDFS needs to coincide with the same command that was used to upgrade it.
> HDFS supports two types of upgrades:
> 1. -rollingUpgrade started (allows downgrading so that there is no data loss. Downgrade is either "-rollingUpgrade started" or "-rollingUpgrade downgrade")
> 2. -upgrade (only allows rollback, so any data that was added since the upgrade will be lost. Rollback is done with "-rollback" command).
>
> Because Express Upgrade will support only these paths, we will use option 1 for all cases.
> HDP 2.1->2.3 (only upgrade)
> HDP 2.2->2.3 (both upgrade and downgrade)
> HDP 2.2->2.2.* (both upgrade and downgrade)
> HDP 2.3->2.3.* (both upgrade and downgrade)
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py f941164
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py ade2ec4
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py c8c057d
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 0f96d03
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 3a518cf
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml d6328dc
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 2f807fd
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 51ccf59
>
> Diff: https://reviews.apache.org/r/40004/diff/
>
>
> Testing
> -------
>
> Tested in the following scenarios.
> HDP 2.1->2.3:
> Express Upgrade: setup with and without HA
>
> HDP 2.2->2.3:
> Express Upgrade: setup with and without HA, followed upgrade and then downgrade.
> Rolling Upgrade: setup with HA, followed upgrade and then downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 40004: AMBARI-13755. Express Upgrade: Failed to
Downgrade Namenode
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40004/#review105400
-----------------------------------------------------------
Ship it!
Ship It!
- Dmitro Lisnichenko
On Nov. 6, 2015, 4:52 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40004/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2015, 4:52 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jayush Luniya, Nate Cole, and Swapan Shridhar.
>
>
> Bugs: AMBARI-13755
> https://issues.apache.org/jira/browse/AMBARI-13755
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Setup:
>
> * Express Upgrade on HDP 2.2.6.0 -> HDP 2.3.0.0
> * Services: HDFS + ZK
> * Upgrade completes successfully
> * Downgrade after reaching Finalize Upgrade
> * Downgrade blocked at Namenode restart
>
> This is happening because the downgrade command to HDFS needs to coincide with the same command that was used to upgrade it.
> HDFS supports two types of upgrades:
> 1. -rollingUpgrade started (allows downgrading so that there is no data loss. Downgrade is either "-rollingUpgrade started" or "-rollingUpgrade downgrade")
> 2. -upgrade (only allows rollback, so any data that was added since the upgrade will be lost. Rollback is done with "-rollback" command).
>
> Because Express Upgrade will support only these paths, we will use option 1 for all cases.
> HDP 2.1->2.3 (only upgrade)
> HDP 2.2->2.3 (both upgrade and downgrade)
> HDP 2.2->2.2.* (both upgrade and downgrade)
> HDP 2.3->2.3.* (both upgrade and downgrade)
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py f941164
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py ade2ec4
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py c8c057d
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 0f96d03
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 3a518cf
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml d6328dc
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 2f807fd
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 51ccf59
>
> Diff: https://reviews.apache.org/r/40004/diff/
>
>
> Testing
> -------
>
> Tested in the following scenarios.
> HDP 2.1->2.3:
> Express Upgrade: setup with and without HA
>
> HDP 2.2->2.3:
> Express Upgrade: setup with and without HA, followed upgrade and then downgrade.
> Rolling Upgrade: setup with HA, followed upgrade and then downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 40004: AMBARI-13755. Express Upgrade: Failed to
Downgrade Namenode
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40004/#review105353
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py (line 104)
<https://reviews.apache.org/r/40004/#comment163916>
This is the main change in logic, to always use -rollingUpgrade [started|downgrade]
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py (line 219)
<https://reviews.apache.org/r/40004/#comment163915>
We can't call conf-select when downgrading on EU because the symlinks were already unlinked, so that /etc/hadoop/conf is now a flat dir.
ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml
<https://reviews.apache.org/r/40004/#comment163911>
This is not being used.
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml
<https://reviews.apache.org/r/40004/#comment163912>
This is not being used.
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml
<https://reviews.apache.org/r/40004/#comment163913>
This is not being used.
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml
<https://reviews.apache.org/r/40004/#comment163914>
This is not being used.
- Alejandro Fernandez
On Nov. 6, 2015, 2:52 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40004/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2015, 2:52 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jayush Luniya, Nate Cole, and Swapan Shridhar.
>
>
> Bugs: AMBARI-13755
> https://issues.apache.org/jira/browse/AMBARI-13755
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Setup:
>
> * Express Upgrade on HDP 2.2.6.0 -> HDP 2.3.0.0
> * Services: HDFS + ZK
> * Upgrade completes successfully
> * Downgrade after reaching Finalize Upgrade
> * Downgrade blocked at Namenode restart
>
> This is happening because the downgrade command to HDFS needs to coincide with the same command that was used to upgrade it.
> HDFS supports two types of upgrades:
> 1. -rollingUpgrade started (allows downgrading so that there is no data loss. Downgrade is either "-rollingUpgrade started" or "-rollingUpgrade downgrade")
> 2. -upgrade (only allows rollback, so any data that was added since the upgrade will be lost. Rollback is done with "-rollback" command).
>
> Because Express Upgrade will support only these paths, we will use option 1 for all cases.
> HDP 2.1->2.3 (only upgrade)
> HDP 2.2->2.3 (both upgrade and downgrade)
> HDP 2.2->2.2.* (both upgrade and downgrade)
> HDP 2.3->2.3.* (both upgrade and downgrade)
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py f941164
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py ade2ec4
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py c8c057d
> ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 0f96d03
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml 3a518cf
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml d6328dc
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml 2f807fd
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 51ccf59
>
> Diff: https://reviews.apache.org/r/40004/diff/
>
>
> Testing
> -------
>
> Tested in the following scenarios.
> HDP 2.1->2.3:
> Express Upgrade: setup with and without HA
>
> HDP 2.2->2.3:
> Express Upgrade: setup with and without HA, followed upgrade and then downgrade.
> Rolling Upgrade: setup with HA, followed upgrade and then downgrade.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>