You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2017/01/05 21:47:25 UTC

Review Request 55230: HOU Fails To Restart NameNode in non-HA Cluster

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55230/
-----------------------------------------------------------

Review request for Ambari, Dmytro Grinenko and Nate Cole.


Bugs: AMBARI-19344
    https://issues.apache.org/jira/browse/AMBARI-19344


Repository: ambari


Description
-------

When restarting NameNode during an HOU, NN throws an error because it was improperly put into rollingUpgrade mode.

*Steps*
# Deploy HDP-2.5.0.0 with Ambari-2.5.0.0-547 build (non-HA cluster)
# Start Host Ordered Upgrade to HDP-2.5.3
 
*Result:* Error at pre Upgrade HDFS step

```
PREPARE rolling upgrade ...
17/01/03 17:07:51 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.rollingUpgrade over null. Not retrying because try once and fail.
org.apache.hadoop.ipc.RemoteException(java.io.IOException): Safe mode should be turned ON in order to create namespace image.
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startRollingUpgradeInternalForNonHA(FSNamesystem.java:7839)

rollingUpgrade: Safe mode should be turned ON in order to create namespace image.
```


Diffs
-----

  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 23119f0 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 86f68e5 
  ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_namenode.py 23119f0 
  ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py 86f68e5 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml f6480bf 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/host-ordered-upgrade.xml 1b29af3 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2627ddf 

Diff: https://reviews.apache.org/r/55230/diff/


Testing
-------

----------------------------------------------------------------------
Total run:1157
Total errors:0
Total failures:0
OK
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 50.822 s
[INFO] Finished at: 2017-01-05T16:29:21-05:00
[INFO] Final Memory: 17M/310M
[INFO] ------------------------------------------------------------------------


Thanks,

Jonathan Hurley


Re: Review Request 55230: HOU Fails To Restart NameNode in non-HA Cluster

Posted by Dmytro Grinenko <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55230/#review160732
-----------------------------------------------------------


Ship it!




Ship It!

- Dmytro Grinenko


On Jan. 6, 2017, 2:21 p.m., Jonathan Hurley wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55230/
> -----------------------------------------------------------
> 
> (Updated Jan. 6, 2017, 2:21 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko and Nate Cole.
> 
> 
> Bugs: AMBARI-19344
>     https://issues.apache.org/jira/browse/AMBARI-19344
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> When restarting NameNode during an HOU, NN throws an error because it was improperly put into rollingUpgrade mode.
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari-2.5.0.0-547 build (non-HA cluster)
> # Start Host Ordered Upgrade to HDP-2.5.3
>  
> *Result:* Error at pre Upgrade HDFS step
> 
> ```
> PREPARE rolling upgrade ...
> 17/01/03 17:07:51 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.rollingUpgrade over null. Not retrying because try once and fail.
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): Safe mode should be turned ON in order to create namespace image.
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startRollingUpgradeInternalForNonHA(FSNamesystem.java:7839)
> 
> rollingUpgrade: Safe mode should be turned ON in order to create namespace image.
> ```
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml f6480bf 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/host-ordered-upgrade.xml 1b29af3 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2627ddf 
> 
> Diff: https://reviews.apache.org/r/55230/diff/
> 
> 
> Testing
> -------
> 
> ----------------------------------------------------------------------
> Total run:1157
> Total errors:0
> Total failures:0
> OK
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 50.822 s
> [INFO] Finished at: 2017-01-05T16:29:21-05:00
> [INFO] Final Memory: 17M/310M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>


Re: Review Request 55230: HOU Fails To Restart NameNode in non-HA Cluster

Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55230/#review160737
-----------------------------------------------------------


Ship it!




Ship It!

- Alejandro Fernandez


On Jan. 6, 2017, 2:21 p.m., Jonathan Hurley wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55230/
> -----------------------------------------------------------
> 
> (Updated Jan. 6, 2017, 2:21 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko and Nate Cole.
> 
> 
> Bugs: AMBARI-19344
>     https://issues.apache.org/jira/browse/AMBARI-19344
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> When restarting NameNode during an HOU, NN throws an error because it was improperly put into rollingUpgrade mode.
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari-2.5.0.0-547 build (non-HA cluster)
> # Start Host Ordered Upgrade to HDP-2.5.3
>  
> *Result:* Error at pre Upgrade HDFS step
> 
> ```
> PREPARE rolling upgrade ...
> 17/01/03 17:07:51 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.rollingUpgrade over null. Not retrying because try once and fail.
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): Safe mode should be turned ON in order to create namespace image.
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startRollingUpgradeInternalForNonHA(FSNamesystem.java:7839)
> 
> rollingUpgrade: Safe mode should be turned ON in order to create namespace image.
> ```
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml f6480bf 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/host-ordered-upgrade.xml 1b29af3 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2627ddf 
> 
> Diff: https://reviews.apache.org/r/55230/diff/
> 
> 
> Testing
> -------
> 
> ----------------------------------------------------------------------
> Total run:1157
> Total errors:0
> Total failures:0
> OK
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 50.822 s
> [INFO] Finished at: 2017-01-05T16:29:21-05:00
> [INFO] Final Memory: 17M/310M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>


Re: Review Request 55230: HOU Fails To Restart NameNode in non-HA Cluster

Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55230/
-----------------------------------------------------------

(Updated Jan. 6, 2017, 9:21 a.m.)


Review request for Ambari, Dmytro Grinenko and Nate Cole.


Bugs: AMBARI-19344
    https://issues.apache.org/jira/browse/AMBARI-19344


Repository: ambari


Description
-------

When restarting NameNode during an HOU, NN throws an error because it was improperly put into rollingUpgrade mode.

*Steps*
# Deploy HDP-2.5.0.0 with Ambari-2.5.0.0-547 build (non-HA cluster)
# Start Host Ordered Upgrade to HDP-2.5.3
 
*Result:* Error at pre Upgrade HDFS step

```
PREPARE rolling upgrade ...
17/01/03 17:07:51 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.rollingUpgrade over null. Not retrying because try once and fail.
org.apache.hadoop.ipc.RemoteException(java.io.IOException): Safe mode should be turned ON in order to create namespace image.
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startRollingUpgradeInternalForNonHA(FSNamesystem.java:7839)

rollingUpgrade: Safe mode should be turned ON in order to create namespace image.
```


Diffs (updated)
-----

  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 23119f0 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 86f68e5 
  ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_namenode.py 23119f0 
  ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py 86f68e5 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml f6480bf 
  ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/host-ordered-upgrade.xml 1b29af3 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2627ddf 

Diff: https://reviews.apache.org/r/55230/diff/


Testing
-------

----------------------------------------------------------------------
Total run:1157
Total errors:0
Total failures:0
OK
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 50.822 s
[INFO] Finished at: 2017-01-05T16:29:21-05:00
[INFO] Final Memory: 17M/310M
[INFO] ------------------------------------------------------------------------


Thanks,

Jonathan Hurley


Re: Review Request 55230: HOU Fails To Restart NameNode in non-HA Cluster

Posted by Jonathan Hurley <jh...@hortonworks.com>.

> On Jan. 6, 2017, 2:29 a.m., Dmytro Grinenko wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py, line 22
> > <https://reviews.apache.org/r/55230/diff/1/?file=1597766#file1597766line22>
> >
> >     why not import via coma in one line(more pythonic) or for example:
> >     
> >      from ambari_commons import constants
> >       .....
> >      
> >       constants.UPGRADE_TYPE_NON_ROLLING

Sure, I can change that.


- Jonathan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55230/#review160687
-----------------------------------------------------------


On Jan. 5, 2017, 4:47 p.m., Jonathan Hurley wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55230/
> -----------------------------------------------------------
> 
> (Updated Jan. 5, 2017, 4:47 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko and Nate Cole.
> 
> 
> Bugs: AMBARI-19344
>     https://issues.apache.org/jira/browse/AMBARI-19344
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> When restarting NameNode during an HOU, NN throws an error because it was improperly put into rollingUpgrade mode.
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari-2.5.0.0-547 build (non-HA cluster)
> # Start Host Ordered Upgrade to HDP-2.5.3
>  
> *Result:* Error at pre Upgrade HDFS step
> 
> ```
> PREPARE rolling upgrade ...
> 17/01/03 17:07:51 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.rollingUpgrade over null. Not retrying because try once and fail.
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): Safe mode should be turned ON in order to create namespace image.
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startRollingUpgradeInternalForNonHA(FSNamesystem.java:7839)
> 
> rollingUpgrade: Safe mode should be turned ON in order to create namespace image.
> ```
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml f6480bf 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/host-ordered-upgrade.xml 1b29af3 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2627ddf 
> 
> Diff: https://reviews.apache.org/r/55230/diff/
> 
> 
> Testing
> -------
> 
> ----------------------------------------------------------------------
> Total run:1157
> Total errors:0
> Total failures:0
> OK
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 50.822 s
> [INFO] Finished at: 2017-01-05T16:29:21-05:00
> [INFO] Final Memory: 17M/310M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>


Re: Review Request 55230: HOU Fails To Restart NameNode in non-HA Cluster

Posted by Dmytro Grinenko <ha...@gmail.com>.

> On Jan. 6, 2017, 7:29 a.m., Dmytro Grinenko wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py, line 22
> > <https://reviews.apache.org/r/55230/diff/1/?file=1597766#file1597766line22>
> >
> >     why not import via coma in one line(more pythonic) or for example:
> >     
> >      from ambari_commons import constants
> >       .....
> >      
> >       constants.UPGRADE_TYPE_NON_ROLLING
> 
> Jonathan Hurley wrote:
>     Sure, I can change that.
> 
> Jonathan Hurley wrote:
>     I just thought that it was more readable as this:
>     
>     if upgrade_type == UPGRADE_TYPE_ROLLING
>     
>     vs
>     
>     if upgrade_type == constants.UPGRADE_TYPE_ROLLING
>     
>     Do you think the 2nd option is clearer?

second options allow to not extend import each time when new constant were added.


- Dmytro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55230/#review160687
-----------------------------------------------------------


On Jan. 6, 2017, 2:21 p.m., Jonathan Hurley wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55230/
> -----------------------------------------------------------
> 
> (Updated Jan. 6, 2017, 2:21 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko and Nate Cole.
> 
> 
> Bugs: AMBARI-19344
>     https://issues.apache.org/jira/browse/AMBARI-19344
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> When restarting NameNode during an HOU, NN throws an error because it was improperly put into rollingUpgrade mode.
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari-2.5.0.0-547 build (non-HA cluster)
> # Start Host Ordered Upgrade to HDP-2.5.3
>  
> *Result:* Error at pre Upgrade HDFS step
> 
> ```
> PREPARE rolling upgrade ...
> 17/01/03 17:07:51 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.rollingUpgrade over null. Not retrying because try once and fail.
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): Safe mode should be turned ON in order to create namespace image.
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startRollingUpgradeInternalForNonHA(FSNamesystem.java:7839)
> 
> rollingUpgrade: Safe mode should be turned ON in order to create namespace image.
> ```
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml f6480bf 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/host-ordered-upgrade.xml 1b29af3 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2627ddf 
> 
> Diff: https://reviews.apache.org/r/55230/diff/
> 
> 
> Testing
> -------
> 
> ----------------------------------------------------------------------
> Total run:1157
> Total errors:0
> Total failures:0
> OK
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 50.822 s
> [INFO] Finished at: 2017-01-05T16:29:21-05:00
> [INFO] Final Memory: 17M/310M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>


Re: Review Request 55230: HOU Fails To Restart NameNode in non-HA Cluster

Posted by Jonathan Hurley <jh...@hortonworks.com>.

> On Jan. 6, 2017, 2:29 a.m., Dmytro Grinenko wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py, line 22
> > <https://reviews.apache.org/r/55230/diff/1/?file=1597766#file1597766line22>
> >
> >     why not import via coma in one line(more pythonic) or for example:
> >     
> >      from ambari_commons import constants
> >       .....
> >      
> >       constants.UPGRADE_TYPE_NON_ROLLING
> 
> Jonathan Hurley wrote:
>     Sure, I can change that.

I just thought that it was more readable as this:

if upgrade_type == UPGRADE_TYPE_ROLLING

vs

if upgrade_type == constants.UPGRADE_TYPE_ROLLING

Do you think the 2nd option is clearer?


- Jonathan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55230/#review160687
-----------------------------------------------------------


On Jan. 5, 2017, 4:47 p.m., Jonathan Hurley wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55230/
> -----------------------------------------------------------
> 
> (Updated Jan. 5, 2017, 4:47 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko and Nate Cole.
> 
> 
> Bugs: AMBARI-19344
>     https://issues.apache.org/jira/browse/AMBARI-19344
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> When restarting NameNode during an HOU, NN throws an error because it was improperly put into rollingUpgrade mode.
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari-2.5.0.0-547 build (non-HA cluster)
> # Start Host Ordered Upgrade to HDP-2.5.3
>  
> *Result:* Error at pre Upgrade HDFS step
> 
> ```
> PREPARE rolling upgrade ...
> 17/01/03 17:07:51 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.rollingUpgrade over null. Not retrying because try once and fail.
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): Safe mode should be turned ON in order to create namespace image.
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startRollingUpgradeInternalForNonHA(FSNamesystem.java:7839)
> 
> rollingUpgrade: Safe mode should be turned ON in order to create namespace image.
> ```
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml f6480bf 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/host-ordered-upgrade.xml 1b29af3 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2627ddf 
> 
> Diff: https://reviews.apache.org/r/55230/diff/
> 
> 
> Testing
> -------
> 
> ----------------------------------------------------------------------
> Total run:1157
> Total errors:0
> Total failures:0
> OK
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 50.822 s
> [INFO] Finished at: 2017-01-05T16:29:21-05:00
> [INFO] Final Memory: 17M/310M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>


Re: Review Request 55230: HOU Fails To Restart NameNode in non-HA Cluster

Posted by Dmytro Grinenko <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55230/#review160687
-----------------------------------------------------------




ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py (line 22)
<https://reviews.apache.org/r/55230/#comment231898>

    why not import via coma in one line(more pythonic) or for example:
    
     from ambari_commons import constants
      .....
     
      constants.UPGRADE_TYPE_NON_ROLLING


- Dmytro Grinenko


On Jan. 5, 2017, 9:47 p.m., Jonathan Hurley wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55230/
> -----------------------------------------------------------
> 
> (Updated Jan. 5, 2017, 9:47 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko and Nate Cole.
> 
> 
> Bugs: AMBARI-19344
>     https://issues.apache.org/jira/browse/AMBARI-19344
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> When restarting NameNode during an HOU, NN throws an error because it was improperly put into rollingUpgrade mode.
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari-2.5.0.0-547 build (non-HA cluster)
> # Start Host Ordered Upgrade to HDP-2.5.3
>  
> *Result:* Error at pre Upgrade HDFS step
> 
> ```
> PREPARE rolling upgrade ...
> 17/01/03 17:07:51 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.rollingUpgrade over null. Not retrying because try once and fail.
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): Safe mode should be turned ON in order to create namespace image.
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startRollingUpgradeInternalForNonHA(FSNamesystem.java:7839)
> 
> rollingUpgrade: Safe mode should be turned ON in order to create namespace image.
> ```
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml f6480bf 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/host-ordered-upgrade.xml 1b29af3 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2627ddf 
> 
> Diff: https://reviews.apache.org/r/55230/diff/
> 
> 
> Testing
> -------
> 
> ----------------------------------------------------------------------
> Total run:1157
> Total errors:0
> Total failures:0
> OK
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 50.822 s
> [INFO] Finished at: 2017-01-05T16:29:21-05:00
> [INFO] Final Memory: 17M/310M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>


Re: Review Request 55230: HOU Fails To Restart NameNode in non-HA Cluster

Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55230/#review160718
-----------------------------------------------------------


Ship it!




Ship It!

- Nate Cole


On Jan. 5, 2017, 4:47 p.m., Jonathan Hurley wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55230/
> -----------------------------------------------------------
> 
> (Updated Jan. 5, 2017, 4:47 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko and Nate Cole.
> 
> 
> Bugs: AMBARI-19344
>     https://issues.apache.org/jira/browse/AMBARI-19344
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> When restarting NameNode during an HOU, NN throws an error because it was improperly put into rollingUpgrade mode.
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari-2.5.0.0-547 build (non-HA cluster)
> # Start Host Ordered Upgrade to HDP-2.5.3
>  
> *Result:* Error at pre Upgrade HDFS step
> 
> ```
> PREPARE rolling upgrade ...
> 17/01/03 17:07:51 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.rollingUpgrade over null. Not retrying because try once and fail.
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): Safe mode should be turned ON in order to create namespace image.
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startRollingUpgradeInternalForNonHA(FSNamesystem.java:7839)
> 
> rollingUpgrade: Safe mode should be turned ON in order to create namespace image.
> ```
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml f6480bf 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/host-ordered-upgrade.xml 1b29af3 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2627ddf 
> 
> Diff: https://reviews.apache.org/r/55230/diff/
> 
> 
> Testing
> -------
> 
> ----------------------------------------------------------------------
> Total run:1157
> Total errors:0
> Total failures:0
> OK
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 50.822 s
> [INFO] Finished at: 2017-01-05T16:29:21-05:00
> [INFO] Final Memory: 17M/310M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>


Re: Review Request 55230: HOU Fails To Restart NameNode in non-HA Cluster

Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55230/#review160635
-----------------------------------------------------------




ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py (line 125)
<https://reviews.apache.org/r/55230/#comment231773>

    Most changes are like this one - just trying to use constants...



ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py (lines 199 - 204)
<https://reviews.apache.org/r/55230/#comment231772>

    This logic was weird, but it caused conf-select to not be called for HOU



ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml 
<https://reviews.apache.org/r/55230/#comment231771>

    This, and the one below it, were the biggest causes of the problem. NN restarts during HOU should be treated as normal restarts - no need to place NN into a "mode" or even create a marker file.


- Jonathan Hurley


On Jan. 5, 2017, 4:47 p.m., Jonathan Hurley wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55230/
> -----------------------------------------------------------
> 
> (Updated Jan. 5, 2017, 4:47 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko and Nate Cole.
> 
> 
> Bugs: AMBARI-19344
>     https://issues.apache.org/jira/browse/AMBARI-19344
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> When restarting NameNode during an HOU, NN throws an error because it was improperly put into rollingUpgrade mode.
> 
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari-2.5.0.0-547 build (non-HA cluster)
> # Start Host Ordered Upgrade to HDP-2.5.3
>  
> *Result:* Error at pre Upgrade HDFS step
> 
> ```
> PREPARE rolling upgrade ...
> 17/01/03 17:07:51 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.rollingUpgrade over null. Not retrying because try once and fail.
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): Safe mode should be turned ON in order to create namespace image.
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startRollingUpgradeInternalForNonHA(FSNamesystem.java:7839)
> 
> rollingUpgrade: Safe mode should be turned ON in order to create namespace image.
> ```
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_namenode.py 23119f0 
>   ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py 86f68e5 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml f6480bf 
>   ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/host-ordered-upgrade.xml 1b29af3 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 2627ddf 
> 
> Diff: https://reviews.apache.org/r/55230/diff/
> 
> 
> Testing
> -------
> 
> ----------------------------------------------------------------------
> Total run:1157
> Total errors:0
> Total failures:0
> OK
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 50.822 s
> [INFO] Finished at: 2017-01-05T16:29:21-05:00
> [INFO] Final Memory: 17M/310M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>