You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2015/01/23 02:48:32 UTC
Review Request 30202: RU - Fix hardcoded Namenode address and JMX port
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/
-----------------------------------------------------------
Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Bugs: AMBARI-9289
https://issues.apache.org/jira/browse/AMBARI-9289
Repository: ambari
Description
-------
Java-side:
UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
Python-side:
When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
See journalnode_upgrade.py
E.g.,
dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e66675fbf1289cd7b797784f14bd8d631f1bf
ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804acea3a888036822aa2ec9789f536418d6
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 746a04ace69a4d8b1c66215901d351f8a2395ef9
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c7c1d5b4896a8ab1b90611caef0baced20
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3b5e3de9084bd8282104c1642237ae5184
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java e3d8619df3a6f0f579b8e592f0ac6bdf51be610f
ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac6435e0876c946101582856b5a4a6e19ef5dd
ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 4e85130d3538f0d76a093a7e0cbcdefec4f6088b
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8fe1255d1d8a3b8a11ae1b6d17b2dc779e
ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
Diff: https://reviews.apache.org/r/30202/diff/
Testing
-------
Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
The second item is for the Journalnode restart to query JMX correctly.
Waiting for unit-test results.
Thanks,
Alejandro Fernandez
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review70794
-----------------------------------------------------------
Ship it!
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
<https://reviews.apache.org/r/30202/#comment116229>
This is still an issue, no? Should be
`Execute(format("{kinit_path_local} -kt {hdfs_user_keytab} {hdfs_principal_name}"))`
- Jonathan Hurley
On Feb. 2, 2015, 8:15 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2015, 8:15 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/script/config_dictionary.py ab05778
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 2414214
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json 0686c57
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-jmx.json 037397a
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-jmx.json b26df20
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-active.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-standby.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 9a227e7
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
> I also tested with Hbase and ResourceManager.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/
-----------------------------------------------------------
(Updated Feb. 3, 2015, 1:15 a.m.)
Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Changes
-------
Changed kinit to sudo as params.hdfs_user
Bugs: AMBARI-9289
https://issues.apache.org/jira/browse/AMBARI-9289
Repository: ambari
Description
-------
Java-side:
UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
Python-side:
When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
See journalnode_upgrade.py
E.g.,
dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/script/config_dictionary.py ab05778
ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 2414214
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json 0686c57
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-jmx.json 037397a
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-jmx.json b26df20
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-active.json PRE-CREATION
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-standby.json PRE-CREATION
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 9a227e7
Diff: https://reviews.apache.org/r/30202/diff/
Testing
-------
Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
The second item is for the Journalnode restart to query JMX correctly.
I also tested with Hbase and ResourceManager.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:14.455s
[INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
[INFO] Final Memory: 50M/758M
[INFO] ------------------------------------------------------------------------
Thanks,
Alejandro Fernandez
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/
-----------------------------------------------------------
(Updated Feb. 3, 2015, 12:38 a.m.)
Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Changes
-------
Addressed Jonathan's comments.
Bugs: AMBARI-9289
https://issues.apache.org/jira/browse/AMBARI-9289
Repository: ambari
Description
-------
Java-side:
UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
Python-side:
When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
See journalnode_upgrade.py
E.g.,
dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/script/config_dictionary.py ab05778
ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 2414214
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json 0686c57
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-jmx.json 037397a
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-jmx.json b26df20
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-active.json PRE-CREATION
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-standby.json PRE-CREATION
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 9a227e7
Diff: https://reviews.apache.org/r/30202/diff/
Testing
-------
Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
The second item is for the Journalnode restart to query JMX correctly.
I also tested with Hbase and ResourceManager.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:14.455s
[INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
[INFO] Final Memory: 50M/758M
[INFO] ------------------------------------------------------------------------
Thanks,
Alejandro Fernandez
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Feb. 3, 2015, 12:28 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py, line 82
> > <https://reviews.apache.org/r/30202/diff/4/?file=844619#file844619line82>
> >
> > I'm surprised that this works. Can we always be certain that the bare name of the hdfs principal is an actual user?
In a kerberized environment, it will be hdfs@REAL.COM, which isn't actually a system user.
> On Feb. 3, 2015, 12:28 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py, line 80
> > <https://reviews.apache.org/r/30202/diff/4/?file=844621#file844621line80>
> >
> > Does `{params.foo}` work here? The pattern that's used through the codebase is simply `{foo}`
It does not.
> On Feb. 3, 2015, 12:28 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java, line 168
> > <https://reviews.apache.org/r/30202/diff/3-4/?file=832576#file832576line168>
> >
> > Log the full exception with stack trace so we can tell where this message came from?
Will do.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review70653
-----------------------------------------------------------
On Feb. 3, 2015, 12:09 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Feb. 3, 2015, 12:09 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/script/config_dictionary.py ab05778
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 2414214
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json 0686c57
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-jmx.json 037397a
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-jmx.json b26df20
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-active.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-standby.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 9a227e7
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
> I also tested with Hbase and ResourceManager.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Feb. 3, 2015, 12:28 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py, line 82
> > <https://reviews.apache.org/r/30202/diff/4/?file=844619#file844619line82>
> >
> > I'm surprised that this works. Can we always be certain that the bare name of the hdfs principal is an actual user?
>
> Alejandro Fernandez wrote:
> In a kerberized environment, it will be hdfs@REAL.COM, which isn't actually a system user.
>
> Jonathan Hurley wrote:
> Right, but you're stripping off the `hdfs` part. What if it's `foo@REAL.com`; are we guaranteed that we can execute commands as the `foo` user?
So I checked with Dilli from the security team. The keytab file access is a principal@REALM, and the principal must exist as a user on the box.
The python code first tries to sudo su {user}, and then run the kinit command. Technically, we can use either, but it's clearer if we run the command as the regular hdfs_user. I'll make this change right now.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review70653
-----------------------------------------------------------
On Feb. 3, 2015, 12:38 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Feb. 3, 2015, 12:38 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/script/config_dictionary.py ab05778
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 2414214
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json 0686c57
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-jmx.json 037397a
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-jmx.json b26df20
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-active.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-standby.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 9a227e7
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
> I also tested with Hbase and ResourceManager.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Feb. 2, 2015, 7:28 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py, line 82
> > <https://reviews.apache.org/r/30202/diff/4/?file=844619#file844619line82>
> >
> > I'm surprised that this works. Can we always be certain that the bare name of the hdfs principal is an actual user?
>
> Alejandro Fernandez wrote:
> In a kerberized environment, it will be hdfs@REAL.COM, which isn't actually a system user.
Right, but you're stripping off the `hdfs` part. What if it's `foo@REAL.com`; are we guaranteed that we can execute commands as the `foo` user?
> On Feb. 2, 2015, 7:28 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py, line 80
> > <https://reviews.apache.org/r/30202/diff/4/?file=844621#file844621line80>
> >
> > Does `{params.foo}` work here? The pattern that's used through the codebase is simply `{foo}`
>
> Alejandro Fernandez wrote:
> It does not.
OK, so if `format("{params.foo}")` does not work, then we need to change it to `format("{foo}")`, right? I didn't see that change in the latest patch.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review70653
-----------------------------------------------------------
On Feb. 2, 2015, 7:38 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2015, 7:38 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/script/config_dictionary.py ab05778
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 2414214
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json 0686c57
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-jmx.json 037397a
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-jmx.json b26df20
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-active.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-standby.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 9a227e7
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
> I also tested with Hbase and ResourceManager.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review70653
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment115970>
Log the full exception with stack trace so we can tell where this message came from?
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py
<https://reviews.apache.org/r/30202/#comment115988>
kerberos_domain can never be None since you're defaulting it to '' above.
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
<https://reviews.apache.org/r/30202/#comment115991>
I'm surprised that this works. Can we always be certain that the bare name of the hdfs principal is an actual user?
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
<https://reviews.apache.org/r/30202/#comment115984>
Same comment as above on hdfs_bare_principal_name
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
<https://reviews.apache.org/r/30202/#comment115975>
Does `{params.foo}` work here? The pattern that's used through the codebase is simply `{foo}`
- Jonathan Hurley
On Feb. 2, 2015, 7:09 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2015, 7:09 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/script/config_dictionary.py ab05778
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 2414214
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json 0686c57
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-jmx.json 037397a
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-jmx.json b26df20
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-active.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-standby.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 9a227e7
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
> I also tested with Hbase and ResourceManager.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/
-----------------------------------------------------------
(Updated Feb. 3, 2015, 12:09 a.m.)
Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Changes
-------
Fixed unit tests after revision #4 went in.
Bugs: AMBARI-9289
https://issues.apache.org/jira/browse/AMBARI-9289
Repository: ambari
Description
-------
Java-side:
UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
Python-side:
When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
See journalnode_upgrade.py
E.g.,
dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/script/config_dictionary.py ab05778
ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 2414214
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json 0686c57
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-jmx.json 037397a
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-jmx.json b26df20
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-active.json PRE-CREATION
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-standby.json PRE-CREATION
ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 9a227e7
Diff: https://reviews.apache.org/r/30202/diff/
Testing
-------
Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
The second item is for the Journalnode restart to query JMX correctly.
I also tested with Hbase and ResourceManager.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:14.455s
[INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
[INFO] Final Memory: 50M/758M
[INFO] ------------------------------------------------------------------------
Thanks,
Alejandro Fernandez
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/
-----------------------------------------------------------
(Updated Feb. 2, 2015, 10:37 p.m.)
Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Bugs: AMBARI-9289
https://issues.apache.org/jira/browse/AMBARI-9289
Repository: ambari
Description
-------
Java-side:
UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
Python-side:
When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
See journalnode_upgrade.py
E.g.,
dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
Diff: https://reviews.apache.org/r/30202/diff/
Testing (updated)
-------
Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
The second item is for the Journalnode restart to query JMX correctly.
I also tested with Hbase and ResourceManager.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:14.455s
[INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
[INFO] Final Memory: 50M/758M
[INFO] ------------------------------------------------------------------------
Thanks,
Alejandro Fernandez
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/
-----------------------------------------------------------
(Updated Feb. 2, 2015, 10:36 p.m.)
Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Changes
-------
Tested on a Kerberized cluster, and fixed hacks for HBase and RM.
Bugs: AMBARI-9289
https://issues.apache.org/jira/browse/AMBARI-9289
Repository: ambari
Description
-------
Java-side:
UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
Python-side:
When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
See journalnode_upgrade.py
E.g.,
dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
Diff: https://reviews.apache.org/r/30202/diff/
Testing
-------
Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
The second item is for the Journalnode restart to query JMX correctly.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:14.455s
[INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
[INFO] Final Memory: 50M/758M
[INFO] ------------------------------------------------------------------------
Thanks,
Alejandro Fernandez
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review69539
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment114229>
Why not address them now?
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment114230>
Why not address them now?
- Nate Cole
On Jan. 23, 2015, 7:57 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2015, 7:57 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e66675fbf1289cd7b797784f14bd8d631f1bf
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804acea3a888036822aa2ec9789f536418d6
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 746a04ace69a4d8b1c66215901d351f8a2395ef9
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c7c1d5b4896a8ab1b90611caef0baced20
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3b5e3de9084bd8282104c1642237ae5184
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java e3d8619df3a6f0f579b8e592f0ac6bdf51be610f
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac6435e0876c946101582856b5a4a6e19ef5dd
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 4e85130d3538f0d76a093a7e0cbcdefec4f6088b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 4d120b12d361a37177015cb1be894fef83462064
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8fe1255d1d8a3b8a11ae1b6d17b2dc779e
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089a1c3a465e87eabc2f1bd7832d62f9b0a7
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review69793
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On Jan. 23, 2015, 7:57 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2015, 7:57 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e66675fbf1289cd7b797784f14bd8d631f1bf
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804acea3a888036822aa2ec9789f536418d6
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 746a04ace69a4d8b1c66215901d351f8a2395ef9
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c7c1d5b4896a8ab1b90611caef0baced20
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3b5e3de9084bd8282104c1642237ae5184
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java e3d8619df3a6f0f579b8e592f0ac6bdf51be610f
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac6435e0876c946101582856b5a4a6e19ef5dd
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 4e85130d3538f0d76a093a7e0cbcdefec4f6088b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 4d120b12d361a37177015cb1be894fef83462064
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8fe1255d1d8a3b8a11ae1b6d17b2dc779e
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089a1c3a465e87eabc2f1bd7832d62f9b0a7
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/
-----------------------------------------------------------
(Updated Jan. 24, 2015, 12:57 a.m.)
Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Changes
-------
Fixed issues reported by Jonathan.
Bugs: AMBARI-9289
https://issues.apache.org/jira/browse/AMBARI-9289
Repository: ambari
Description
-------
Java-side:
UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
Python-side:
When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
See journalnode_upgrade.py
E.g.,
dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e66675fbf1289cd7b797784f14bd8d631f1bf
ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804acea3a888036822aa2ec9789f536418d6
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 746a04ace69a4d8b1c66215901d351f8a2395ef9
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c7c1d5b4896a8ab1b90611caef0baced20
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3b5e3de9084bd8282104c1642237ae5184
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java e3d8619df3a6f0f579b8e592f0ac6bdf51be610f
ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac6435e0876c946101582856b5a4a6e19ef5dd
ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 4e85130d3538f0d76a093a7e0cbcdefec4f6088b
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 4d120b12d361a37177015cb1be894fef83462064
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8fe1255d1d8a3b8a11ae1b6d17b2dc779e
ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089a1c3a465e87eabc2f1bd7832d62f9b0a7
ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
Diff: https://reviews.apache.org/r/30202/diff/
Testing
-------
Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
The second item is for the Journalnode restart to query JMX correctly.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:14.455s
[INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
[INFO] Final Memory: 50M/758M
[INFO] ------------------------------------------------------------------------
Thanks,
Alejandro Fernandez
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Jan. 23, 2015, 9:05 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py, line 84
> > <https://reviews.apache.org/r/30202/diff/2/?file=831417#file831417line84>
> >
> > This uses url2lib instead of curl; Alerts just got slammed with this when the environment was kerberized. Without a proper keytab, the URL request would hit a 401 Unauthorized error.
> >
> > Do we use this pattern a lot? I'd think it would fail all over the place.
I don't think you fixed this. url2lib won't work when requesting JMX resources in a kerberized environment.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review69388
-----------------------------------------------------------
On Jan. 23, 2015, 7:57 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2015, 7:57 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e66675fbf1289cd7b797784f14bd8d631f1bf
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804acea3a888036822aa2ec9789f536418d6
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 746a04ace69a4d8b1c66215901d351f8a2395ef9
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c7c1d5b4896a8ab1b90611caef0baced20
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3b5e3de9084bd8282104c1642237ae5184
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java e3d8619df3a6f0f579b8e592f0ac6bdf51be610f
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac6435e0876c946101582856b5a4a6e19ef5dd
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 4e85130d3538f0d76a093a7e0cbcdefec4f6088b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 4d120b12d361a37177015cb1be894fef83462064
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8fe1255d1d8a3b8a11ae1b6d17b2dc779e
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089a1c3a465e87eabc2f1bd7832d62f9b0a7
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Jan. 23, 2015, 9:05 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py, line 84
> > <https://reviews.apache.org/r/30202/diff/2/?file=831417#file831417line84>
> >
> > This uses url2lib instead of curl; Alerts just got slammed with this when the environment was kerberized. Without a proper keytab, the URL request would hit a 401 Unauthorized error.
> >
> > Do we use this pattern a lot? I'd think it would fail all over the place.
>
> Jonathan Hurley wrote:
> I don't think you fixed this. url2lib won't work when requesting JMX resources in a kerberized environment.
>
> Alejandro Fernandez wrote:
> Jonathan, I ended up testing this on a kerberized cluster and it worked.
I'm really surprised by this. Can you explain to me why it works? url2lib doesn't support SPNEGO, so even if you had a valid ticket, the library would not be able to negotiate with the server to use it. I'm not saying that I don't believe you here, but I want to make sure we understand why it works and that we're not missing something. Since Ambari is running as root, the ticket would need to be for the root user, which it's definitely not.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review69388
-----------------------------------------------------------
On Feb. 2, 2015, 7:09 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2015, 7:09 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/script/config_dictionary.py ab05778
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 2414214
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json 0686c57
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-jmx.json 037397a
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-jmx.json b26df20
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-active.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-standby.json PRE-CREATION
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 9a227e7
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
> I also tested with Hbase and ResourceManager.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Jan. 23, 2015, 2:05 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py, line 84
> > <https://reviews.apache.org/r/30202/diff/2/?file=831417#file831417line84>
> >
> > This uses url2lib instead of curl; Alerts just got slammed with this when the environment was kerberized. Without a proper keytab, the URL request would hit a 401 Unauthorized error.
> >
> > Do we use this pattern a lot? I'd think it would fail all over the place.
>
> Jonathan Hurley wrote:
> I don't think you fixed this. url2lib won't work when requesting JMX resources in a kerberized environment.
Jonathan, I ended up testing this on a kerberized cluster and it worked.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review69388
-----------------------------------------------------------
On Feb. 2, 2015, 10:36 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2015, 10:36 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review69388
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment114081>
configHelper param
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment114082>
configHelper param
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment114087>
This method is geared to only work for NN HA (2 or more name nodes). This line should be removed and the callers of this method should not call it unless there are 2 hosts.
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment114083>
As this doesn't change, it could just be a static. In fact, it seems as though this kind of configuration type (and the others) should all be public statics somewhere.
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment114084>
Inline return; change to {} for better visual structure.
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment114085>
Inline return; use {}
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment114086>
"HTTPS_ONLY" should probably be a public static somewhere.
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py
<https://reviews.apache.org/r/30202/#comment114088>
This uses url2lib instead of curl; Alerts just got slammed with this when the environment was kerberized. Without a proper keytab, the URL request would hit a 401 Unauthorized error.
Do we use this pattern a lot? I'd think it would fail all over the place.
- Jonathan Hurley
On Jan. 22, 2015, 9:13 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Jan. 22, 2015, 9:13 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e66675fbf1289cd7b797784f14bd8d631f1bf
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804acea3a888036822aa2ec9789f536418d6
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 746a04ace69a4d8b1c66215901d351f8a2395ef9
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c7c1d5b4896a8ab1b90611caef0baced20
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3b5e3de9084bd8282104c1642237ae5184
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java e3d8619df3a6f0f579b8e592f0ac6bdf51be610f
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac6435e0876c946101582856b5a4a6e19ef5dd
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 4e85130d3538f0d76a093a7e0cbcdefec4f6088b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8fe1255d1d8a3b8a11ae1b6d17b2dc779e
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089a1c3a465e87eabc2f1bd7832d62f9b0a7
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review69374
-----------------------------------------------------------
Ship it!
Ship It!
- Dmitro Lisnichenko
On Jan. 23, 2015, 2:13 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2015, 2:13 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e66675fbf1289cd7b797784f14bd8d631f1bf
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804acea3a888036822aa2ec9789f536418d6
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 746a04ace69a4d8b1c66215901d351f8a2395ef9
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c7c1d5b4896a8ab1b90611caef0baced20
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3b5e3de9084bd8282104c1642237ae5184
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java e3d8619df3a6f0f579b8e592f0ac6bdf51be610f
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac6435e0876c946101582856b5a4a6e19ef5dd
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 4e85130d3538f0d76a093a7e0cbcdefec4f6088b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8fe1255d1d8a3b8a11ae1b6d17b2dc779e
> ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089a1c3a465e87eabc2f1bd7832d62f9b0a7
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/
-----------------------------------------------------------
(Updated Jan. 23, 2015, 2:13 a.m.)
Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Changes
-------
Fixed unit test.
Bugs: AMBARI-9289
https://issues.apache.org/jira/browse/AMBARI-9289
Repository: ambari
Description
-------
Java-side:
UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
Python-side:
When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
See journalnode_upgrade.py
E.g.,
dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e66675fbf1289cd7b797784f14bd8d631f1bf
ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804acea3a888036822aa2ec9789f536418d6
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 746a04ace69a4d8b1c66215901d351f8a2395ef9
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c7c1d5b4896a8ab1b90611caef0baced20
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3b5e3de9084bd8282104c1642237ae5184
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java e3d8619df3a6f0f579b8e592f0ac6bdf51be610f
ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac6435e0876c946101582856b5a4a6e19ef5dd
ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 4e85130d3538f0d76a093a7e0cbcdefec4f6088b
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8fe1255d1d8a3b8a11ae1b6d17b2dc779e
ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089a1c3a465e87eabc2f1bd7832d62f9b0a7
ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
Diff: https://reviews.apache.org/r/30202/diff/
Testing
-------
Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
The second item is for the Journalnode restart to query JMX correctly.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:14.455s
[INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
[INFO] Final Memory: 50M/758M
[INFO] ------------------------------------------------------------------------
Thanks,
Alejandro Fernandez
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/
-----------------------------------------------------------
(Updated Jan. 23, 2015, 2:11 a.m.)
Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
Bugs: AMBARI-9289
https://issues.apache.org/jira/browse/AMBARI-9289
Repository: ambari
Description
-------
Java-side:
UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
Python-side:
When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
See journalnode_upgrade.py
E.g.,
dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e66675fbf1289cd7b797784f14bd8d631f1bf
ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804acea3a888036822aa2ec9789f536418d6
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 746a04ace69a4d8b1c66215901d351f8a2395ef9
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c7c1d5b4896a8ab1b90611caef0baced20
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3b5e3de9084bd8282104c1642237ae5184
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java e3d8619df3a6f0f579b8e592f0ac6bdf51be610f
ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac6435e0876c946101582856b5a4a6e19ef5dd
ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 4e85130d3538f0d76a093a7e0cbcdefec4f6088b
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8fe1255d1d8a3b8a11ae1b6d17b2dc779e
ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
Diff: https://reviews.apache.org/r/30202/diff/
Testing (updated)
-------
Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
The second item is for the Journalnode restart to query JMX correctly.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:14.455s
[INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
[INFO] Final Memory: 50M/758M
[INFO] ------------------------------------------------------------------------
Thanks,
Alejandro Fernandez
Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
port
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review69328
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment113993>
ConfigHelper is now needed by MasterHostResolver
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment113994>
There's similar logic in the python scripts.
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
<https://reviews.apache.org/r/30202/#comment113995>
Default is for HTTP protocol instead of HTTPS.
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
<https://reviews.apache.org/r/30202/#comment113997>
Broke this out since I needed to call it from MasterHostResolver
ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java
<https://reviews.apache.org/r/30202/#comment113998>
See unit test below.
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py
<https://reviews.apache.org/r/30202/#comment114002>
This is where the magic happens.
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py
<https://reviews.apache.org/r/30202/#comment113999>
@Jonathan, at some point we can refactor the alerts python files to use this class.
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py
<https://reviews.apache.org/r/30202/#comment114000>
The __str__ comes in pretty handy.
- Alejandro Fernandez
On Jan. 23, 2015, 1:48 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2015, 1:48 a.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
>
>
> Bugs: AMBARI-9289
> https://issues.apache.org/jira/browse/AMBARI-9289
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
>
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e66675fbf1289cd7b797784f14bd8d631f1bf
> ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804acea3a888036822aa2ec9789f536418d6
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 746a04ace69a4d8b1c66215901d351f8a2395ef9
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c7c1d5b4896a8ab1b90611caef0baced20
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3b5e3de9084bd8282104c1642237ae5184
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java e3d8619df3a6f0f579b8e592f0ac6bdf51be610f
> ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac6435e0876c946101582856b5a4a6e19ef5dd
> ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 4e85130d3538f0d76a093a7e0cbcdefec4f6088b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8fe1255d1d8a3b8a11ae1b6d17b2dc779e
> ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/30202/diff/
>
>
> Testing
> -------
>
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
>
> Waiting for unit-test results.
>
>
> Thanks,
>
> Alejandro Fernandez
>
>