You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2017/01/10 21:24:13 UTC
Review Request 55391: Role Command Order For HOU Is Different For Some
Components
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55391/
-----------------------------------------------------------
Review request for Ambari, Nate Cole, Robert Levas, and Sid Wagle.
Bugs: AMBARI-19448
https://issues.apache.org/jira/browse/AMBARI-19448
Repository: ambari
Description
-------
As of AMBARI-19435, HOU now uses the role command ordering to determine how to start components on a host. This is because some components (such as NodeManager and DataNode) require their masters to be online first in order to register correctly. When co-located on the same host, masters and slaves must be started in the correct order.
However, some ordering defined on the stack will not work for HOU. For example, the stack doesn't enforce that NameNode start before DataNode. This is because in a normal start command, the DataNode can wait for up to 30 minutes for the NameNode to come online. During an upgrade, the upgrade logic waits to ensure that the DataNode is online and registered before moving on. Since the role ordering allows DataNode to start first, this start always fails during an upgrade.
In order to allow this custom ordering, a new section was added for upgrades to the role_command_order.json.
```
{
"_comment" : "Record format:",
"_comment" : "blockedRole-blockedCommand: [blockerRole1-blockerCommand1 ... ",
"general_deps" : {
...
},
"host_ordered_upgrade" : {
"DATANODE-START": ["NAMENODE-START"]
}
```
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java 7cf197b
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java cebc1b7
ambari-server/src/main/java/org/apache/ambari/server/state/stack/StackRoleCommandOrder.java b660ec1
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java abb2aab
ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 5a9825d
ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java f0a40c9
ambari-server/src/test/resources/stacks/HDP/2.2.0/role_command_order.json faaee6e
Diff: https://reviews.apache.org/r/55391/diff/
Testing
-------
PENDING...
Thanks,
Jonathan Hurley
Re: Review Request 55391: Role Command Order For HOU Is Different For
Some Components
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Jan. 11, 2017, 9:41 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java, line 405
> > <https://reviews.apache.org/r/55391/diff/1/?file=1601797#file1601797line405>
> >
> > This isn't a deep copy of the map. Will that be an issue?
No, it shouldn't be. A new map is created, but it takes the references to the Set<> instances along with it. That's OK since there's no way to directly modify that Set. Instead, you can call initalize() again which will blow away the references for the cloned instance.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55391/#review161121
-----------------------------------------------------------
On Jan. 10, 2017, 4:24 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55391/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2017, 4:24 p.m.)
>
>
> Review request for Ambari, Nate Cole, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-19448
> https://issues.apache.org/jira/browse/AMBARI-19448
>
>
> Repository: ambari
>
>
> Description
> -------
>
> As of AMBARI-19435, HOU now uses the role command ordering to determine how to start components on a host. This is because some components (such as NodeManager and DataNode) require their masters to be online first in order to register correctly. When co-located on the same host, masters and slaves must be started in the correct order.
>
> However, some ordering defined on the stack will not work for HOU. For example, the stack doesn't enforce that NameNode start before DataNode. This is because in a normal start command, the DataNode can wait for up to 30 minutes for the NameNode to come online. During an upgrade, the upgrade logic waits to ensure that the DataNode is online and registered before moving on. Since the role ordering allows DataNode to start first, this start always fails during an upgrade.
>
> In order to allow this custom ordering, a new section was added for upgrades to the role_command_order.json.
>
> ```
> {
> "_comment" : "Record format:",
> "_comment" : "blockedRole-blockedCommand: [blockerRole1-blockerCommand1 ... ",
> "general_deps" : {
> ...
> },
> "host_ordered_upgrade" : {
> "DATANODE-START": ["NAMENODE-START"]
> }
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java 7cf197b
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java cebc1b7
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/StackRoleCommandOrder.java b660ec1
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java abb2aab
> ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 5a9825d
> ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java f0a40c9
> ambari-server/src/test/resources/stacks/HDP/2.2.0/role_command_order.json faaee6e
>
> Diff: https://reviews.apache.org/r/55391/diff/
>
>
> Testing
> -------
>
> PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 55391: Role Command Order For HOU Is Different For
Some Components
Posted by Robert Levas <rl...@hortonworks.com>.
> On Jan. 11, 2017, 9:41 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java, line 405
> > <https://reviews.apache.org/r/55391/diff/1/?file=1601797#file1601797line405>
> >
> > This isn't a deep copy of the map. Will that be an issue?
>
> Jonathan Hurley wrote:
> No, it shouldn't be. A new map is created, but it takes the references to the Set<> instances along with it. That's OK since there's no way to directly modify that Set. Instead, you can call initalize() again which will blow away the references for the cloned instance.
Thanks for the clarification. `Clone` can be rather ambiguous.
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55391/#review161121
-----------------------------------------------------------
On Jan. 10, 2017, 4:24 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55391/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2017, 4:24 p.m.)
>
>
> Review request for Ambari, Nate Cole, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-19448
> https://issues.apache.org/jira/browse/AMBARI-19448
>
>
> Repository: ambari
>
>
> Description
> -------
>
> As of AMBARI-19435, HOU now uses the role command ordering to determine how to start components on a host. This is because some components (such as NodeManager and DataNode) require their masters to be online first in order to register correctly. When co-located on the same host, masters and slaves must be started in the correct order.
>
> However, some ordering defined on the stack will not work for HOU. For example, the stack doesn't enforce that NameNode start before DataNode. This is because in a normal start command, the DataNode can wait for up to 30 minutes for the NameNode to come online. During an upgrade, the upgrade logic waits to ensure that the DataNode is online and registered before moving on. Since the role ordering allows DataNode to start first, this start always fails during an upgrade.
>
> In order to allow this custom ordering, a new section was added for upgrades to the role_command_order.json.
>
> ```
> {
> "_comment" : "Record format:",
> "_comment" : "blockedRole-blockedCommand: [blockerRole1-blockerCommand1 ... ",
> "general_deps" : {
> ...
> },
> "host_ordered_upgrade" : {
> "DATANODE-START": ["NAMENODE-START"]
> }
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java 7cf197b
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java cebc1b7
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/StackRoleCommandOrder.java b660ec1
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java abb2aab
> ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 5a9825d
> ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java f0a40c9
> ambari-server/src/test/resources/stacks/HDP/2.2.0/role_command_order.json faaee6e
>
> Diff: https://reviews.apache.org/r/55391/diff/
>
>
> Testing
> -------
>
> PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 55391: Role Command Order For HOU Is Different For
Some Components
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55391/#review161121
-----------------------------------------------------------
Fix it, then Ship it!
Ship It!
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java (line 373)
<https://reviews.apache.org/r/55391/#comment232360>
This isn't a deep copy of the map. Will that be an issue?
- Robert Levas
On Jan. 10, 2017, 4:24 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55391/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2017, 4:24 p.m.)
>
>
> Review request for Ambari, Nate Cole, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-19448
> https://issues.apache.org/jira/browse/AMBARI-19448
>
>
> Repository: ambari
>
>
> Description
> -------
>
> As of AMBARI-19435, HOU now uses the role command ordering to determine how to start components on a host. This is because some components (such as NodeManager and DataNode) require their masters to be online first in order to register correctly. When co-located on the same host, masters and slaves must be started in the correct order.
>
> However, some ordering defined on the stack will not work for HOU. For example, the stack doesn't enforce that NameNode start before DataNode. This is because in a normal start command, the DataNode can wait for up to 30 minutes for the NameNode to come online. During an upgrade, the upgrade logic waits to ensure that the DataNode is online and registered before moving on. Since the role ordering allows DataNode to start first, this start always fails during an upgrade.
>
> In order to allow this custom ordering, a new section was added for upgrades to the role_command_order.json.
>
> ```
> {
> "_comment" : "Record format:",
> "_comment" : "blockedRole-blockedCommand: [blockerRole1-blockerCommand1 ... ",
> "general_deps" : {
> ...
> },
> "host_ordered_upgrade" : {
> "DATANODE-START": ["NAMENODE-START"]
> }
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java 7cf197b
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java cebc1b7
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/StackRoleCommandOrder.java b660ec1
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java abb2aab
> ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 5a9825d
> ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java f0a40c9
> ambari-server/src/test/resources/stacks/HDP/2.2.0/role_command_order.json faaee6e
>
> Diff: https://reviews.apache.org/r/55391/diff/
>
>
> Testing
> -------
>
> PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 55391: Role Command Order For HOU Is Different For
Some Components
Posted by Sid Wagle <sw...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55391/#review161276
-----------------------------------------------------------
Ship it!
Ship It!
- Sid Wagle
On Jan. 10, 2017, 9:24 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55391/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2017, 9:24 p.m.)
>
>
> Review request for Ambari, Nate Cole, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-19448
> https://issues.apache.org/jira/browse/AMBARI-19448
>
>
> Repository: ambari
>
>
> Description
> -------
>
> As of AMBARI-19435, HOU now uses the role command ordering to determine how to start components on a host. This is because some components (such as NodeManager and DataNode) require their masters to be online first in order to register correctly. When co-located on the same host, masters and slaves must be started in the correct order.
>
> However, some ordering defined on the stack will not work for HOU. For example, the stack doesn't enforce that NameNode start before DataNode. This is because in a normal start command, the DataNode can wait for up to 30 minutes for the NameNode to come online. During an upgrade, the upgrade logic waits to ensure that the DataNode is online and registered before moving on. Since the role ordering allows DataNode to start first, this start always fails during an upgrade.
>
> In order to allow this custom ordering, a new section was added for upgrades to the role_command_order.json.
>
> ```
> {
> "_comment" : "Record format:",
> "_comment" : "blockedRole-blockedCommand: [blockerRole1-blockerCommand1 ... ",
> "general_deps" : {
> ...
> },
> "host_ordered_upgrade" : {
> "DATANODE-START": ["NAMENODE-START"]
> }
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java 7cf197b
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java cebc1b7
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/StackRoleCommandOrder.java b660ec1
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java abb2aab
> ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 5a9825d
> ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java f0a40c9
> ambari-server/src/test/resources/stacks/HDP/2.2.0/role_command_order.json faaee6e
>
> Diff: https://reviews.apache.org/r/55391/diff/
>
>
> Testing
> -------
>
> PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 55391: Role Command Order For HOU Is Different For
Some Components
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Jan. 11, 2017, 12:18 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java, lines 119-124
> > <https://reviews.apache.org/r/55391/diff/1/?file=1601797#file1601797line119>
> >
> > Sigh :) Can we make jira to move RCO to something real instead of this parsing madness.
Yes, yes we can. I did like your idea about making it a specific structure:
```
"overrides": {
"host_ordered_upgrade": {
"DATANODE-START": "merge|replace|\u2026"
```
Which then then be applied via rco.appyOverrides("host_ordered_upgrade") ... but that would require some extra code to parse and for the caller to know it was an override (as opposed to a section).
And at the end of the day, if we're going to make this a real DTO which can be properly parsed, that work would have been throw away. Anyway - I'll make the Jira.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55391/#review161272
-----------------------------------------------------------
On Jan. 10, 2017, 4:24 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55391/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2017, 4:24 p.m.)
>
>
> Review request for Ambari, Nate Cole, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-19448
> https://issues.apache.org/jira/browse/AMBARI-19448
>
>
> Repository: ambari
>
>
> Description
> -------
>
> As of AMBARI-19435, HOU now uses the role command ordering to determine how to start components on a host. This is because some components (such as NodeManager and DataNode) require their masters to be online first in order to register correctly. When co-located on the same host, masters and slaves must be started in the correct order.
>
> However, some ordering defined on the stack will not work for HOU. For example, the stack doesn't enforce that NameNode start before DataNode. This is because in a normal start command, the DataNode can wait for up to 30 minutes for the NameNode to come online. During an upgrade, the upgrade logic waits to ensure that the DataNode is online and registered before moving on. Since the role ordering allows DataNode to start first, this start always fails during an upgrade.
>
> In order to allow this custom ordering, a new section was added for upgrades to the role_command_order.json.
>
> ```
> {
> "_comment" : "Record format:",
> "_comment" : "blockedRole-blockedCommand: [blockerRole1-blockerCommand1 ... ",
> "general_deps" : {
> ...
> },
> "host_ordered_upgrade" : {
> "DATANODE-START": ["NAMENODE-START"]
> }
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java 7cf197b
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java cebc1b7
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/StackRoleCommandOrder.java b660ec1
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java abb2aab
> ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 5a9825d
> ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java f0a40c9
> ambari-server/src/test/resources/stacks/HDP/2.2.0/role_command_order.json faaee6e
>
> Diff: https://reviews.apache.org/r/55391/diff/
>
>
> Testing
> -------
>
> PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 55391: Role Command Order For HOU Is Different For
Some Components
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55391/#review161272
-----------------------------------------------------------
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java (lines 70 - 71)
<https://reviews.apache.org/r/55391/#comment232507>
Thank you!
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java (lines 111 - 116)
<https://reviews.apache.org/r/55391/#comment232508>
Sigh :) Can we make jira to move RCO to something real instead of this parsing madness.
- Nate Cole
On Jan. 10, 2017, 4:24 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55391/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2017, 4:24 p.m.)
>
>
> Review request for Ambari, Nate Cole, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-19448
> https://issues.apache.org/jira/browse/AMBARI-19448
>
>
> Repository: ambari
>
>
> Description
> -------
>
> As of AMBARI-19435, HOU now uses the role command ordering to determine how to start components on a host. This is because some components (such as NodeManager and DataNode) require their masters to be online first in order to register correctly. When co-located on the same host, masters and slaves must be started in the correct order.
>
> However, some ordering defined on the stack will not work for HOU. For example, the stack doesn't enforce that NameNode start before DataNode. This is because in a normal start command, the DataNode can wait for up to 30 minutes for the NameNode to come online. During an upgrade, the upgrade logic waits to ensure that the DataNode is online and registered before moving on. Since the role ordering allows DataNode to start first, this start always fails during an upgrade.
>
> In order to allow this custom ordering, a new section was added for upgrades to the role_command_order.json.
>
> ```
> {
> "_comment" : "Record format:",
> "_comment" : "blockedRole-blockedCommand: [blockerRole1-blockerCommand1 ... ",
> "general_deps" : {
> ...
> },
> "host_ordered_upgrade" : {
> "DATANODE-START": ["NAMENODE-START"]
> }
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java 7cf197b
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java cebc1b7
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/StackRoleCommandOrder.java b660ec1
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java abb2aab
> ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 5a9825d
> ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java f0a40c9
> ambari-server/src/test/resources/stacks/HDP/2.2.0/role_command_order.json faaee6e
>
> Diff: https://reviews.apache.org/r/55391/diff/
>
>
> Testing
> -------
>
> PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 55391: Role Command Order For HOU Is Different For
Some Components
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55391/#review161120
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java (lines 105 - 119)
<https://reviews.apache.org/r/55391/#comment232356>
It made no sense to have getter/setters for every section, especially when nobody used them - made it a linked set of strings instead.
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java (lines 115 - 117)
<https://reviews.apache.org/r/55391/#comment232357>
So there was no way to easily "override" existing dependencies. I went with this approach, which appends -OVERRIDE to the order, since it followed the convention here of parsing and was easy.
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java (lines 150 - 154)
<https://reviews.apache.org/r/55391/#comment232358>
As mentioned above, there's no need for all of this if/else nonesense - just add the sections specified by the LinkedHashSet
ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json (lines 22 - 27)
<https://reviews.apache.org/r/55391/#comment232359>
This is the big change here - adding a new section with the -OVERRIDE option.
- Jonathan Hurley
On Jan. 10, 2017, 4:24 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55391/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2017, 4:24 p.m.)
>
>
> Review request for Ambari, Nate Cole, Robert Levas, and Sid Wagle.
>
>
> Bugs: AMBARI-19448
> https://issues.apache.org/jira/browse/AMBARI-19448
>
>
> Repository: ambari
>
>
> Description
> -------
>
> As of AMBARI-19435, HOU now uses the role command ordering to determine how to start components on a host. This is because some components (such as NodeManager and DataNode) require their masters to be online first in order to register correctly. When co-located on the same host, masters and slaves must be started in the correct order.
>
> However, some ordering defined on the stack will not work for HOU. For example, the stack doesn't enforce that NameNode start before DataNode. This is because in a normal start command, the DataNode can wait for up to 30 minutes for the NameNode to come online. During an upgrade, the upgrade logic waits to ensure that the DataNode is online and registered before moving on. Since the role ordering allows DataNode to start first, this start always fails during an upgrade.
>
> In order to allow this custom ordering, a new section was added for upgrades to the role_command_order.json.
>
> ```
> {
> "_comment" : "Record format:",
> "_comment" : "blockedRole-blockedCommand: [blockerRole1-blockerCommand1 ... ",
> "general_deps" : {
> ...
> },
> "host_ordered_upgrade" : {
> "DATANODE-START": ["NAMENODE-START"]
> }
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java 7cf197b
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java cebc1b7
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/StackRoleCommandOrder.java b660ec1
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java abb2aab
> ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 5a9825d
> ambari-server/src/test/java/org/apache/ambari/server/metadata/RoleCommandOrderTest.java f0a40c9
> ambari-server/src/test/resources/stacks/HDP/2.2.0/role_command_order.json faaee6e
>
> Diff: https://reviews.apache.org/r/55391/diff/
>
>
> Testing
> -------
>
> PENDING...
>
>
> Thanks,
>
> Jonathan Hurley
>
>