You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2015/10/20 00:02:50 UTC
Re: Review Request 39418: If Ambari Server is not part of the cluster,
basic operations throw 500 (NPE) when Kerberos is enabled (even if it is
Manual Kerberos)
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39418/#review103165
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java (line 789)
<https://reviews.apache.org/r/39418/#comment161128>
I don't think that `hasCommandsInProgress` can handle a null hostname. If it can, we should doc that it can be null and needs to be handled as such.
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java (lines 145 - 147)
<https://reviews.apache.org/r/39418/#comment161129>
Odd that you're checking for getHostEntity and then extracting getHostId. You should probably just null-check the getHostId()
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java (lines 161 - 164)
<https://reviews.apache.org/r/39418/#comment161132>
I don't think this works. The alter column stuff only worries about data types and length; it doesn't write out nullable/non-nullable. You'll probably need to use GenericDbmsHelper.writeSetNullableString()
- Jonathan Hurley
On Oct. 17, 2015, 8:37 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39418/
> -----------------------------------------------------------
>
> (Updated Oct. 17, 2015, 8:37 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
>
>
> Bugs: AMBARI-13441
> https://issues.apache.org/jira/browse/AMBARI-13441
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a back-port of AMBARI-12506 to Ambari 2.1.3
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java 30da833
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java 1e019b7
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java fec76ab
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 22b2ca4
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java 20b4658
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 27e3377
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java f04c868
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java c005efc
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 670e925
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java a94723f
> ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql bb0c724
> ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql c884d07
> ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7f8b981
> ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 93a5dc8
> ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 621a524
> ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 72d2d25
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/StageTest.java c1bd8bc
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java 798261b
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java 73b1649
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java 069f67d
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c0c9d5e
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/ServerActionExecutorTest.java f9b023f
> ambari-server/src/test/java/org/apache/ambari/server/stageplanner/TestStagePlanner.java c9d7856
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 8d82967
>
> Diff: https://reviews.apache.org/r/39418/diff/
>
>
> Testing
> -------
>
> Manually tested:
> - 1.7.0 -> upgrade to 2.1.3 -> enable kerberos (needed to fix hdfs-site/dfs.namenode.rpc-address and JCE policy on Ambari Server host)
> - 1.7.0 (ambari server host w/o agent) -> upgrade to 2.1.3 -> enable kerberos (needed to fix hdfs-site/dfs.namenode.rpc-address and JCE policy on Ambari Server host)
> - 2.0.2 -> upgrade to 2.1.3 -> enable kerberos
> - 2.0.2 -> enable Kerberos -> upgrade to 2.1.3 -> enable NN HA -> enable RM HA -> RU to HDP 2.3.2.0
> - 2.1.2 -> upgrade to 2.1.3 -> enable kerberos
> - 2.1.2 -> enable Kerberos -> upgrade to 2.1.3
> - 2.1.3 -> enable Kerberos
> - 2.1.3 (ambari server host w/o agent) -> enable kerberos
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 39418: If Ambari Server is not part of the cluster,
basic operations throw 500 (NPE) when Kerberos is enabled (even if it is
Manual Kerberos)
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Oct. 19, 2015, 6:02 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java, lines 161-164
> > <https://reviews.apache.org/r/39418/diff/1/?file=1100540#file1100540line161>
> >
> > I don't think this works. The alter column stuff only worries about data types and length; it doesn't write out nullable/non-nullable. You'll probably need to use GenericDbmsHelper.writeSetNullableString()
>
> Robert Levas wrote:
> In my tests (after upgrading from 1.7.0, 2.0.2) I didn't have any issue with setting the host_id column. Also...
>
> Before the upgrade (Ambari 2.1.0):
> ```
> host_id | bigint | not null | plain |
> ```
>
> After ambari-server upgrade (to Ambari 2.1.3):
> ```
> host_id | bigint | | plain |
> ```
Thanks for the due diligence here. I traced through the spaghetti code that is the database upgrade stuff. It looks like we are explicitely setting this ourselves in the alterColumn:
if (isColumnNullable(tableName, columnInfo.getName()) != columnInfo.isNullable()) {
setColumnNullable(tableName, columnInfo, columnInfo.isNullable());
}
I had only checked the EclipseLink code :)
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39418/#review103165
-----------------------------------------------------------
On Oct. 20, 2015, 1:25 a.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39418/
> -----------------------------------------------------------
>
> (Updated Oct. 20, 2015, 1:25 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
>
>
> Bugs: AMBARI-13441
> https://issues.apache.org/jira/browse/AMBARI-13441
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a back-port of AMBARI-12506 to Ambari 2.1.3
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java 30da833
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java 1e019b7
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java fec76ab
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 22b2ca4
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java 20b4658
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 27e3377
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java f04c868
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java c005efc
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 670e925
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java 0a5a66b
> ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql bb0c724
> ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql c884d07
> ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7f8b981
> ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 93a5dc8
> ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 621a524
> ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 72d2d25
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/StageTest.java c1bd8bc
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java 798261b
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java 73b1649
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java 069f67d
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c0c9d5e
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/ServerActionExecutorTest.java f9b023f
> ambari-server/src/test/java/org/apache/ambari/server/stageplanner/TestStagePlanner.java c9d7856
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 20a3b1c
>
> Diff: https://reviews.apache.org/r/39418/diff/
>
>
> Testing
> -------
>
> Manually tested:
> - 1.7.0 -> upgrade to 2.1.3 -> enable kerberos (needed to fix hdfs-site/dfs.namenode.rpc-address and JCE policy on Ambari Server host)
> - 1.7.0 (ambari server host w/o agent) -> upgrade to 2.1.3 -> enable kerberos (needed to fix hdfs-site/dfs.namenode.rpc-address and JCE policy on Ambari Server host)
> - 2.0.2 -> upgrade to 2.1.3 -> enable kerberos
> - 2.0.2 -> enable Kerberos -> upgrade to 2.1.3 -> enable NN HA -> enable RM HA -> RU to HDP 2.3.2.0
> - 2.1.2 -> upgrade to 2.1.3 -> enable kerberos
> - 2.1.2 -> enable Kerberos -> upgrade to 2.1.3
> - 2.1.3 -> enable Kerberos
> - 2.1.3 (ambari server host w/o agent) -> enable kerberos
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 39418: If Ambari Server is not part of the cluster,
basic operations throw 500 (NPE) when Kerberos is enabled (even if it is
Manual Kerberos)
Posted by Robert Levas <rl...@hortonworks.com>.
> On Oct. 19, 2015, 6:02 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java, line 789
> > <https://reviews.apache.org/r/39418/diff/1/?file=1100532#file1100532line789>
> >
> > I don't think that `hasCommandsInProgress` can handle a null hostname. If it can, we should doc that it can be null and needs to be handled as such.
`hasCommandInProgress` can handled a null hostname. It calls some methods on the passed in `stage` object, each of which will convert a null hostname to `org.apache.ambari.server.actionmanager.Stage#INTERNAL_HOSTNAME` before performing operations.
I will document this.
> On Oct. 19, 2015, 6:02 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java, lines 145-147
> > <https://reviews.apache.org/r/39418/diff/1/?file=1100533#file1100533line145>
> >
> > Odd that you're checking for getHostEntity and then extracting getHostId. You should probably just null-check the getHostId()
Agreed. I should have looked at this more closly when poting the patch to this branch. I will fix.
> On Oct. 19, 2015, 6:02 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java, lines 161-164
> > <https://reviews.apache.org/r/39418/diff/1/?file=1100540#file1100540line161>
> >
> > I don't think this works. The alter column stuff only worries about data types and length; it doesn't write out nullable/non-nullable. You'll probably need to use GenericDbmsHelper.writeSetNullableString()
In my tests (after upgrading from 1.7.0, 2.0.2) I didn't have any issue with setting the host_id column. Also...
Before the upgrade (Ambari 2.1.0):
```
host_id | bigint | not null | plain |
```
After ambari-server upgrade (to Ambari 2.1.3):
```
host_id | bigint | | plain |
```
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39418/#review103165
-----------------------------------------------------------
On Oct. 17, 2015, 8:37 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39418/
> -----------------------------------------------------------
>
> (Updated Oct. 17, 2015, 8:37 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
>
>
> Bugs: AMBARI-13441
> https://issues.apache.org/jira/browse/AMBARI-13441
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This is a back-port of AMBARI-12506 to Ambari 2.1.3
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java 30da833
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java 1e019b7
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java fec76ab
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 22b2ca4
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java 20b4658
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 27e3377
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java f04c868
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java c005efc
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java 670e925
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java a94723f
> ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql bb0c724
> ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql c884d07
> ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7f8b981
> ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 93a5dc8
> ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 621a524
> ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 72d2d25
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/StageTest.java c1bd8bc
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java 798261b
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java 73b1649
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java 069f67d
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java c0c9d5e
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/ServerActionExecutorTest.java f9b023f
> ambari-server/src/test/java/org/apache/ambari/server/stageplanner/TestStagePlanner.java c9d7856
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 8d82967
>
> Diff: https://reviews.apache.org/r/39418/diff/
>
>
> Testing
> -------
>
> Manually tested:
> - 1.7.0 -> upgrade to 2.1.3 -> enable kerberos (needed to fix hdfs-site/dfs.namenode.rpc-address and JCE policy on Ambari Server host)
> - 1.7.0 (ambari server host w/o agent) -> upgrade to 2.1.3 -> enable kerberos (needed to fix hdfs-site/dfs.namenode.rpc-address and JCE policy on Ambari Server host)
> - 2.0.2 -> upgrade to 2.1.3 -> enable kerberos
> - 2.0.2 -> enable Kerberos -> upgrade to 2.1.3 -> enable NN HA -> enable RM HA -> RU to HDP 2.3.2.0
> - 2.1.2 -> upgrade to 2.1.3 -> enable kerberos
> - 2.1.2 -> enable Kerberos -> upgrade to 2.1.3
> - 2.1.3 -> enable Kerberos
> - 2.1.3 (ambari server host w/o agent) -> enable kerberos
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>