You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Nate Cole <nc...@hortonworks.com> on 2017/03/29 20:20:40 UTC
Review Request 58046: yarn resource manager failed to start after
upgrade with unable to construct queue exception.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58046/
-----------------------------------------------------------
Review request for Ambari, Jonathan Hurley and Sumit Mohanty.
Bugs: AMBARI-20622
https://issues.apache.org/jira/browse/AMBARI-20622
Repository: ambari
Description
-------
ordering-policy for queues can only be two possible values in HDP-2.6.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicy.java PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.6.xml c8f93e436f
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml 3c0769a720
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.6.xml 53caa52c6b
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml 66f1a64d9d
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml 0d6f51b2e1
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 44d0a2a1d9
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicyTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/58046/diff/1/
Testing
-------
pending
Thanks,
Nate Cole
Re: Review Request 58046: yarn resource manager failed to start after
upgrade with unable to construct queue exception.
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58046/#review170477
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On March 29, 2017, 8:20 p.m., Nate Cole wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58046/
> -----------------------------------------------------------
>
> (Updated March 29, 2017, 8:20 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Sumit Mohanty.
>
>
> Bugs: AMBARI-20622
> https://issues.apache.org/jira/browse/AMBARI-20622
>
>
> Repository: ambari
>
>
> Description
> -------
>
> ordering-policy for queues can only be two possible values in HDP-2.6.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicy.java PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.6.xml c8f93e436f
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml 3c0769a720
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.6.xml 53caa52c6b
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml 66f1a64d9d
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml 0d6f51b2e1
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 44d0a2a1d9
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicyTest.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/58046/diff/1/
>
>
> Testing
> -------
>
> pending
>
>
> Thanks,
>
> Nate Cole
>
>
Re: Review Request 58046: yarn resource manager failed to start after
upgrade with unable to construct queue exception.
Posted by Nate Cole <nc...@hortonworks.com>.
> On March 29, 2017, 4:29 p.m., Sumit Mohanty wrote:
> > Assuming that the regex pattern is correct, the changes look good.
Still discussing that with the all-knowing Jonathan
- Nate
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58046/#review170473
-----------------------------------------------------------
On March 29, 2017, 4:20 p.m., Nate Cole wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58046/
> -----------------------------------------------------------
>
> (Updated March 29, 2017, 4:20 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Sumit Mohanty.
>
>
> Bugs: AMBARI-20622
> https://issues.apache.org/jira/browse/AMBARI-20622
>
>
> Repository: ambari
>
>
> Description
> -------
>
> ordering-policy for queues can only be two possible values in HDP-2.6.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicy.java PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.6.xml c8f93e436f
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml 3c0769a720
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.6.xml 53caa52c6b
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml 66f1a64d9d
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml 0d6f51b2e1
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 44d0a2a1d9
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicyTest.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/58046/diff/1/
>
>
> Testing
> -------
>
> pending
>
>
> Thanks,
>
> Nate Cole
>
>
Re: Review Request 58046: yarn resource manager failed to start after
upgrade with unable to construct queue exception.
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58046/#review170473
-----------------------------------------------------------
Assuming that the regex pattern is correct, the changes look good.
- Sumit Mohanty
On March 29, 2017, 8:20 p.m., Nate Cole wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58046/
> -----------------------------------------------------------
>
> (Updated March 29, 2017, 8:20 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Sumit Mohanty.
>
>
> Bugs: AMBARI-20622
> https://issues.apache.org/jira/browse/AMBARI-20622
>
>
> Repository: ambari
>
>
> Description
> -------
>
> ordering-policy for queues can only be two possible values in HDP-2.6.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicy.java PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.6.xml c8f93e436f
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml 3c0769a720
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.6.xml 53caa52c6b
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml 66f1a64d9d
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml 0d6f51b2e1
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 44d0a2a1d9
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicyTest.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/58046/diff/1/
>
>
> Testing
> -------
>
> pending
>
>
> Thanks,
>
> Nate Cole
>
>
Re: Review Request 58046: yarn resource manager failed to start after
upgrade with unable to construct queue exception.
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58046/#review170480
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On March 29, 2017, 4:20 p.m., Nate Cole wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58046/
> -----------------------------------------------------------
>
> (Updated March 29, 2017, 4:20 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Sumit Mohanty.
>
>
> Bugs: AMBARI-20622
> https://issues.apache.org/jira/browse/AMBARI-20622
>
>
> Repository: ambari
>
>
> Description
> -------
>
> ordering-policy for queues can only be two possible values in HDP-2.6.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicy.java PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.6.xml c8f93e436f
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml 3c0769a720
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.6.xml 53caa52c6b
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml 66f1a64d9d
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml 0d6f51b2e1
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 44d0a2a1d9
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicyTest.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/58046/diff/1/
>
>
> Testing
> -------
>
> pending
>
>
> Thanks,
>
> Nate Cole
>
>
Re: Review Request 58046: yarn resource manager failed to start after
upgrade with unable to construct queue exception.
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On March 29, 2017, 4:31 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicy.java
> > Lines 74-79 (patched)
> > <https://reviews.apache.org/r/58046/diff/1/?file=1680000#file1680000line74>
> >
> > I think is going to match too much. It would match all:
> >
> > yarn.scheduler.capacity.root.queues
> > yarn.scheduler.capacity.root.hive.queues
> >
> > ...
> >
> > If fifo is OK for leaf queues but not root queues, then can we do this:
> >
> > - Read `yarn.scheduler.capacity.root.queues` and parse each root queue name.
> >
> > - For each queue name, create yarn.scheduler.capacity.root.%s.ordering-policy and set it to `utilization`?
OK - it's root and parents that have to change, so I think this regex is OK.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58046/#review170474
-----------------------------------------------------------
On March 29, 2017, 4:20 p.m., Nate Cole wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58046/
> -----------------------------------------------------------
>
> (Updated March 29, 2017, 4:20 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Sumit Mohanty.
>
>
> Bugs: AMBARI-20622
> https://issues.apache.org/jira/browse/AMBARI-20622
>
>
> Repository: ambari
>
>
> Description
> -------
>
> ordering-policy for queues can only be two possible values in HDP-2.6.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicy.java PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.6.xml c8f93e436f
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml 3c0769a720
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.6.xml 53caa52c6b
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml 66f1a64d9d
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml 0d6f51b2e1
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 44d0a2a1d9
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicyTest.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/58046/diff/1/
>
>
> Testing
> -------
>
> pending
>
>
> Thanks,
>
> Nate Cole
>
>
Re: Review Request 58046: yarn resource manager failed to start after
upgrade with unable to construct queue exception.
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58046/#review170474
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicy.java
Lines 74-79 (patched)
<https://reviews.apache.org/r/58046/#comment243294>
I think is going to match too much. It would match all:
yarn.scheduler.capacity.root.queues
yarn.scheduler.capacity.root.hive.queues
...
If fifo is OK for leaf queues but not root queues, then can we do this:
- Read `yarn.scheduler.capacity.root.queues` and parse each root queue name.
- For each queue name, create yarn.scheduler.capacity.root.%s.ordering-policy and set it to `utilization`?
- Jonathan Hurley
On March 29, 2017, 4:20 p.m., Nate Cole wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58046/
> -----------------------------------------------------------
>
> (Updated March 29, 2017, 4:20 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Sumit Mohanty.
>
>
> Bugs: AMBARI-20622
> https://issues.apache.org/jira/browse/AMBARI-20622
>
>
> Repository: ambari
>
>
> Description
> -------
>
> ordering-policy for queues can only be two possible values in HDP-2.6.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicy.java PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.6.xml c8f93e436f
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml 3c0769a720
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.6.xml 53caa52c6b
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml 66f1a64d9d
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml 0d6f51b2e1
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 44d0a2a1d9
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixCapacitySchedulerOrderingPolicyTest.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/58046/diff/1/
>
>
> Testing
> -------
>
> pending
>
>
> Thanks,
>
> Nate Cole
>
>