You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2015/01/13 19:55:28 UTC
Review Request 29858: Manual Upgrade Tasks Should Be More Explicit And
Parameterized
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29858/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez and Nate Cole.
Bugs: AMBARI-9109
https://issues.apache.org/jira/browse/AMBARI-9109
Repository: ambari
Description
-------
For Hive upgrade, the following manual step was shown:
{noformat}
Run the SQL file at /usr/hdp/$version/hive/scripts/metastore/upgrade to update the Hive Metastore schema.
{noformat}
This is too vague. I could not figure out what to do.
1. It should tell the user exactly which host this needs to be done on.
2. $version should be substituted with the actual directory name.
3. There are 40+ SQL files in the aforementioned "upgrade" folder. What's the exact command that I need to run?
All services that use Manual Tasks as part of the upgrade should be revisited once the parameterization code is completed to ensure that the messages are correct. These include: Hive, Falcon, Oozie, HDFS
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeGroupResourceProvider.java 4df0f9c
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java 46d0dd4
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java b8d8502
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java b453362
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 5882e3e
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java ac30762
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml c110a0b
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 53505f2
ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_bucket_test.xml b777d94
ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml aee1fcf
Diff: https://reviews.apache.org/r/29858/diff/
Testing
-------
New tests written, full tests pending...
Thanks,
Jonathan Hurley
Re: Review Request 29858: Manual Upgrade Tasks Should Be More
Explicit And Parameterized
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Jan. 13, 2015, 2:44 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java, lines 300-301
> > <https://reviews.apache.org/r/29858/diff/1/?file=819657#file819657line300>
> >
> > Maybe in the future we can keep the HostsType object with the StageWrapper, as the call to mhr.getMasterAndHosts() is expensive.
We're lucky in that the manual tasks don't occur for services/components that are expensive :) Maybe we should pre-calculate this ahead of time and store it? Thoughts?
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29858/#review67921
-----------------------------------------------------------
On Jan. 13, 2015, 2:39 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29858/
> -----------------------------------------------------------
>
> (Updated Jan. 13, 2015, 2:39 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-9109
> https://issues.apache.org/jira/browse/AMBARI-9109
>
>
> Repository: ambari
>
>
> Description
> -------
>
> For Hive upgrade, the following manual step was shown:
> {noformat}
> Run the SQL file at /usr/hdp/$version/hive/scripts/metastore/upgrade to update the Hive Metastore schema.
> {noformat}
>
> This is too vague. I could not figure out what to do.
> 1. It should tell the user exactly which host this needs to be done on.
> 2. $version should be substituted with the actual directory name.
> 3. There are 40+ SQL files in the aforementioned "upgrade" folder. What's the exact command that I need to run?
>
> All services that use Manual Tasks as part of the upgrade should be revisited once the parameterization code is completed to ensure that the messages are correct. These include: Hive, Falcon, Oozie, HDFS
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeGroupResourceProvider.java 4df0f9c
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java 46d0dd4
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java b8d8502
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java b453362
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 5882e3e
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java ac30762
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml c110a0b
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 53505f2
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_bucket_test.xml b777d94
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml aee1fcf
>
> Diff: https://reviews.apache.org/r/29858/diff/
>
>
> Testing
> -------
>
> New tests written
>
> Tests run: 2526, Failures: 0, Errors: 0, Skipped: 15
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 29858: Manual Upgrade Tasks Should Be More
Explicit And Parameterized
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29858/#review67921
-----------------------------------------------------------
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java
<https://reviews.apache.org/r/29858/#comment112007>
Maybe in the future we can keep the HostsType object with the StageWrapper, as the call to mhr.getMasterAndHosts() is expensive.
- Nate Cole
On Jan. 13, 2015, 2:39 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29858/
> -----------------------------------------------------------
>
> (Updated Jan. 13, 2015, 2:39 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-9109
> https://issues.apache.org/jira/browse/AMBARI-9109
>
>
> Repository: ambari
>
>
> Description
> -------
>
> For Hive upgrade, the following manual step was shown:
> {noformat}
> Run the SQL file at /usr/hdp/$version/hive/scripts/metastore/upgrade to update the Hive Metastore schema.
> {noformat}
>
> This is too vague. I could not figure out what to do.
> 1. It should tell the user exactly which host this needs to be done on.
> 2. $version should be substituted with the actual directory name.
> 3. There are 40+ SQL files in the aforementioned "upgrade" folder. What's the exact command that I need to run?
>
> All services that use Manual Tasks as part of the upgrade should be revisited once the parameterization code is completed to ensure that the messages are correct. These include: Hive, Falcon, Oozie, HDFS
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeGroupResourceProvider.java 4df0f9c
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java 46d0dd4
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java b8d8502
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java b453362
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 5882e3e
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java ac30762
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml c110a0b
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 53505f2
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_bucket_test.xml b777d94
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml aee1fcf
>
> Diff: https://reviews.apache.org/r/29858/diff/
>
>
> Testing
> -------
>
> New tests written
>
> Tests run: 2526, Failures: 0, Errors: 0, Skipped: 15
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 29858: Manual Upgrade Tasks Should Be More
Explicit And Parameterized
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Jan. 13, 2015, 2:48 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java, line 368
> > <https://reviews.apache.org/r/29858/diff/1/?file=819654#file819654line368>
> >
> > Did you mean to use format function inside here?
No, we should never be formatting strings inside of log statements; the log framework can do that for us.
> On Jan. 13, 2015, 2:48 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java, line 328
> > <https://reviews.apache.org/r/29858/diff/1/?file=819657#file819657line328>
> >
> > Should this join on ", "?
I think so; the idea is that this is a human-readable list of all of the hosts. Since the XML doesn't support structure, then I think the best we can do is host1,host2,host3 etc. What should we use?
> On Jan. 13, 2015, 2:48 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml, line 349
> > <https://reviews.apache.org/r/29858/diff/1/?file=819658#file819658line349>
> >
> > Will the port be configurable?
No, it will not be configurable.
> On Jan. 13, 2015, 2:48 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java, line 512
> > <https://reviews.apache.org/r/29858/diff/1/?file=819656#file819656line512>
> >
> > Should maybe also check that ends with "}}"
Agreed. I'll make this change.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29858/#review67920
-----------------------------------------------------------
On Jan. 13, 2015, 2:39 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29858/
> -----------------------------------------------------------
>
> (Updated Jan. 13, 2015, 2:39 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-9109
> https://issues.apache.org/jira/browse/AMBARI-9109
>
>
> Repository: ambari
>
>
> Description
> -------
>
> For Hive upgrade, the following manual step was shown:
> {noformat}
> Run the SQL file at /usr/hdp/$version/hive/scripts/metastore/upgrade to update the Hive Metastore schema.
> {noformat}
>
> This is too vague. I could not figure out what to do.
> 1. It should tell the user exactly which host this needs to be done on.
> 2. $version should be substituted with the actual directory name.
> 3. There are 40+ SQL files in the aforementioned "upgrade" folder. What's the exact command that I need to run?
>
> All services that use Manual Tasks as part of the upgrade should be revisited once the parameterization code is completed to ensure that the messages are correct. These include: Hive, Falcon, Oozie, HDFS
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeGroupResourceProvider.java 4df0f9c
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java 46d0dd4
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java b8d8502
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java b453362
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 5882e3e
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java ac30762
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml c110a0b
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 53505f2
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_bucket_test.xml b777d94
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml aee1fcf
>
> Diff: https://reviews.apache.org/r/29858/diff/
>
>
> Testing
> -------
>
> New tests written
>
> Tests run: 2526, Failures: 0, Errors: 0, Skipped: 15
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 29858: Manual Upgrade Tasks Should Be More
Explicit And Parameterized
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29858/#review67920
-----------------------------------------------------------
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
<https://reviews.apache.org/r/29858/#comment112005>
Did you mean to use format function inside here?
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
<https://reviews.apache.org/r/29858/#comment112006>
Should maybe also check that ends with "}}"
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java
<https://reviews.apache.org/r/29858/#comment112009>
Should this join on ", "?
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml
<https://reviews.apache.org/r/29858/#comment112010>
Will the port be configurable?
- Alejandro Fernandez
On Jan. 13, 2015, 7:39 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29858/
> -----------------------------------------------------------
>
> (Updated Jan. 13, 2015, 7:39 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-9109
> https://issues.apache.org/jira/browse/AMBARI-9109
>
>
> Repository: ambari
>
>
> Description
> -------
>
> For Hive upgrade, the following manual step was shown:
> {noformat}
> Run the SQL file at /usr/hdp/$version/hive/scripts/metastore/upgrade to update the Hive Metastore schema.
> {noformat}
>
> This is too vague. I could not figure out what to do.
> 1. It should tell the user exactly which host this needs to be done on.
> 2. $version should be substituted with the actual directory name.
> 3. There are 40+ SQL files in the aforementioned "upgrade" folder. What's the exact command that I need to run?
>
> All services that use Manual Tasks as part of the upgrade should be revisited once the parameterization code is completed to ensure that the messages are correct. These include: Hive, Falcon, Oozie, HDFS
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeGroupResourceProvider.java 4df0f9c
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java 46d0dd4
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java b8d8502
> ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java b453362
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 5882e3e
> ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java ac30762
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml c110a0b
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 53505f2
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_bucket_test.xml b777d94
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml aee1fcf
>
> Diff: https://reviews.apache.org/r/29858/diff/
>
>
> Testing
> -------
>
> New tests written
>
> Tests run: 2526, Failures: 0, Errors: 0, Skipped: 15
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 29858: Manual Upgrade Tasks Should Be More
Explicit And Parameterized
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29858/
-----------------------------------------------------------
(Updated Jan. 13, 2015, 2:39 p.m.)
Review request for Ambari, Alejandro Fernandez and Nate Cole.
Bugs: AMBARI-9109
https://issues.apache.org/jira/browse/AMBARI-9109
Repository: ambari
Description
-------
For Hive upgrade, the following manual step was shown:
{noformat}
Run the SQL file at /usr/hdp/$version/hive/scripts/metastore/upgrade to update the Hive Metastore schema.
{noformat}
This is too vague. I could not figure out what to do.
1. It should tell the user exactly which host this needs to be done on.
2. $version should be substituted with the actual directory name.
3. There are 40+ SQL files in the aforementioned "upgrade" folder. What's the exact command that I need to run?
All services that use Manual Tasks as part of the upgrade should be revisited once the parameterization code is completed to ensure that the messages are correct. These include: Hive, Falcon, Oozie, HDFS
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeGroupResourceProvider.java 4df0f9c
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeItemResourceProvider.java 46d0dd4
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java b8d8502
ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java b453362
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 5882e3e
ambari-server/src/main/java/org/apache/ambari/server/state/UpgradeHelper.java ac30762
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml c110a0b
ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 53505f2
ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_bucket_test.xml b777d94
ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml aee1fcf
Diff: https://reviews.apache.org/r/29858/diff/
Testing (updated)
-------
New tests written
Tests run: 2526, Failures: 0, Errors: 0, Skipped: 15
Thanks,
Jonathan Hurley