You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Vitalyi Brodetskyi <vb...@hortonworks.com> on 2017/02/09 18:55:36 UTC
Review Request 56511: To make sure LLAP preemption happens correctly,
should enable
priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-19947
https://issues.apache.org/jira/browse/AMBARI-19947
Repository: ambari
Description
-------
We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
Diff: https://reviews.apache.org/r/56511/diff/
Testing
-------
mvn clean test
Thanks,
Vitalyi Brodetskyi
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/#review164981
-----------------------------------------------------------
Ship it!
Fix the other issue and ship it.
- Sumit Mohanty
On Feb. 9, 2017, 6:55 p.m., Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56511/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2017, 6:55 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-19947
> https://issues.apache.org/jira/browse/AMBARI-19947
>
>
> Repository: ambari
>
>
> Description
> -------
>
> We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
> And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
> So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
> This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
>
> Diff: https://reviews.apache.org/r/56511/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
> On \u041b\u044e\u0442. 9, 2017, 7:09 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f, Sumit Mohanty wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml, line 26
> > <https://reviews.apache.org/r/56511/diff/1/?file=1628669#file1628669line26>
> >
> > Do we need upgrade pack changes to set this property to true when the condition is right?
>
> Sumit Mohanty wrote:
> Oh! a separate JIRA is tracking this.
Sumit Mohanty, i've created separate jira https://hortonworks.jira.com/browse/BUG-74552 for RU/EU changes and assigned it to Dmytro Grinenko. Pay attention that iset "add on upgrade false", i think we don't need it. It's first ambari version where we have HDP-2.6. For fresh install of HDP 2.6 it will appear because it's already added to config and stack advisor. And during RU/EU it will appear too after Dmytro Grinenko's changes. Is it ok?
- Vitalyi
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/#review164976
-----------------------------------------------------------
On \u041b\u044e\u0442. 9, 2017, 6:55 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f, Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56511/
> -----------------------------------------------------------
>
> (Updated \u041b\u044e\u0442. 9, 2017, 6:55 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-19947
> https://issues.apache.org/jira/browse/AMBARI-19947
>
>
> Repository: ambari
>
>
> Description
> -------
>
> We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
> And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
> So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
> This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
>
> Diff: https://reviews.apache.org/r/56511/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Sumit Mohanty <sm...@hortonworks.com>.
> On Feb. 9, 2017, 7:09 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml, line 26
> > <https://reviews.apache.org/r/56511/diff/1/?file=1628669#file1628669line26>
> >
> > Do we need upgrade pack changes to set this property to true when the condition is right?
Oh! a separate JIRA is tracking this.
- Sumit
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/#review164976
-----------------------------------------------------------
On Feb. 9, 2017, 6:55 p.m., Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56511/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2017, 6:55 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-19947
> https://issues.apache.org/jira/browse/AMBARI-19947
>
>
> Repository: ambari
>
>
> Description
> -------
>
> We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
> And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
> So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
> This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
>
> Diff: https://reviews.apache.org/r/56511/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/#review164976
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml (line 26)
<https://reviews.apache.org/r/56511/#comment236812>
Do we need upgrade pack changes to set this property to true when the condition is right?
- Sumit Mohanty
On Feb. 9, 2017, 6:55 p.m., Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56511/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2017, 6:55 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-19947
> https://issues.apache.org/jira/browse/AMBARI-19947
>
>
> Repository: ambari
>
>
> Description
> -------
>
> We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
> And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
> So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
> This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
>
> Diff: https://reviews.apache.org/r/56511/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/#review165052
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On Feb. 9, 2017, 10:51 p.m., Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56511/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2017, 10:51 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-19947
> https://issues.apache.org/jira/browse/AMBARI-19947
>
>
> Repository: ambari
>
>
> Description
> -------
>
> We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
> And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
> So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
> This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/service_advisor.py 30dd474
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
>
> Diff: https://reviews.apache.org/r/56511/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Dmytro Sen <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/#review165113
-----------------------------------------------------------
Ship it!
Ship It!
- Dmytro Sen
On \u0424\u0435\u0432. 10, 2017, 12:40 \u043f.\u043f., Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56511/
> -----------------------------------------------------------
>
> (Updated \u0424\u0435\u0432. 10, 2017, 12:40 \u043f.\u043f.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-19947
> https://issues.apache.org/jira/browse/AMBARI-19947
>
>
> Repository: ambari
>
>
> Description
> -------
>
> We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
> And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
> So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
> This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/service_advisor.py 30dd474
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
> ambari-server/src/main/resources/stacks/HDP/3.0/services/YARN/configuration/yarn-site.xml 0f46d75
>
> Diff: https://reviews.apache.org/r/56511/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/
-----------------------------------------------------------
(Updated \u041b\u044e\u0442. 10, 2017, 12:40 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f)
Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-19947
https://issues.apache.org/jira/browse/AMBARI-19947
Repository: ambari
Description
-------
We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/service_advisor.py 30dd474
ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
ambari-server/src/main/resources/stacks/HDP/3.0/services/YARN/configuration/yarn-site.xml 0f46d75
Diff: https://reviews.apache.org/r/56511/diff/
Testing
-------
mvn clean test
Thanks,
Vitalyi Brodetskyi
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/
-----------------------------------------------------------
(Updated \u041b\u044e\u0442. 9, 2017, 10:51 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f)
Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-19947
https://issues.apache.org/jira/browse/AMBARI-19947
Repository: ambari
Description
-------
We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/service_advisor.py 30dd474
ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
Diff: https://reviews.apache.org/r/56511/diff/
Testing
-------
mvn clean test
Thanks,
Vitalyi Brodetskyi
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
> On \u041b\u044e\u0442. 9, 2017, 7:01 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f, Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml, line 26
> > <https://reviews.apache.org/r/56511/diff/1/?file=1628669#file1628669line26>
> >
> > Also make this change for YARN 3.0.0.3.0, including in its Service Advisor
Why do we need to add these changes for 3.0.0? As i know yarn-site will got this properties by inheritance from HDP-2.6. Similar situation with stack_advisor, in every HDP stack advisor we are calling previous HDP recommendations. Beside this, i don't see stack_advisor.py files at all. Probably we should add separate jira for 3.0.0 to create stack_advisor and test it?
- Vitalyi
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/#review164972
-----------------------------------------------------------
On \u041b\u044e\u0442. 9, 2017, 6:55 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f, Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56511/
> -----------------------------------------------------------
>
> (Updated \u041b\u044e\u0442. 9, 2017, 6:55 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-19947
> https://issues.apache.org/jira/browse/AMBARI-19947
>
>
> Repository: ambari
>
>
> Description
> -------
>
> We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
> And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
> So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
> This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
>
> Diff: https://reviews.apache.org/r/56511/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
> On \u041b\u044e\u0442. 9, 2017, 7:01 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f, Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml, line 26
> > <https://reviews.apache.org/r/56511/diff/1/?file=1628669#file1628669line26>
> >
> > Also make this change for YARN 3.0.0.3.0, including in its Service Advisor
>
> Vitalyi Brodetskyi wrote:
> Why do we need to add these changes for 3.0.0? As i know yarn-site will got this properties by inheritance from HDP-2.6. Similar situation with stack_advisor, in every HDP stack advisor we are calling previous HDP recommendations. Beside this, i don't see stack_advisor.py files at all. Probably we should add separate jira for 3.0.0 to create stack_advisor and test it?
>
> Alejandro Fernandez wrote:
> We've already copied the internals of Stack Advisor for ZK, HDFS, and YARN from HDP 2.0.6-2.6 into Service Advisors. So any further changes we make to those services in older Stack Advisors must also be included in the new Service Advisor.
> YARN has it in common-services/YARN/3.0.0.3.0/service_advisor.py
Task is done but looks like we have some issues in service advisor, because needed property is available but logic with dependency from other property doesn't work, recommendations are not called when i changing it. I will create new jira to investigate it.
- Vitalyi
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/#review164972
-----------------------------------------------------------
On \u041b\u044e\u0442. 9, 2017, 10:51 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f, Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56511/
> -----------------------------------------------------------
>
> (Updated \u041b\u044e\u0442. 9, 2017, 10:51 \u043f\u0456\u0441\u043b\u044f \u043f\u043e\u043b\u0443\u0434\u043d\u044f)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-19947
> https://issues.apache.org/jira/browse/AMBARI-19947
>
>
> Repository: ambari
>
>
> Description
> -------
>
> We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
> And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
> So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
> This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/service_advisor.py 30dd474
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
>
> Diff: https://reviews.apache.org/r/56511/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Feb. 9, 2017, 7:01 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml, line 26
> > <https://reviews.apache.org/r/56511/diff/1/?file=1628669#file1628669line26>
> >
> > Also make this change for YARN 3.0.0.3.0, including in its Service Advisor
>
> Vitalyi Brodetskyi wrote:
> Why do we need to add these changes for 3.0.0? As i know yarn-site will got this properties by inheritance from HDP-2.6. Similar situation with stack_advisor, in every HDP stack advisor we are calling previous HDP recommendations. Beside this, i don't see stack_advisor.py files at all. Probably we should add separate jira for 3.0.0 to create stack_advisor and test it?
We've already copied the internals of Stack Advisor for ZK, HDFS, and YARN from HDP 2.0.6-2.6 into Service Advisors. So any further changes we make to those services in older Stack Advisors must also be included in the new Service Advisor.
YARN has it in common-services/YARN/3.0.0.3.0/service_advisor.py
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/#review164972
-----------------------------------------------------------
On Feb. 9, 2017, 6:55 p.m., Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56511/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2017, 6:55 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-19947
> https://issues.apache.org/jira/browse/AMBARI-19947
>
>
> Repository: ambari
>
>
> Description
> -------
>
> We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
> And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
> So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
> This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
>
> Diff: https://reviews.apache.org/r/56511/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>
Re: Review Request 56511: To make sure LLAP preemption happens
correctly,
should enable priority-utilization.underutilized-preemption.enabled=true
when YARN preemption is enabled
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56511/#review164972
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml (line 26)
<https://reviews.apache.org/r/56511/#comment236807>
Also make this change for YARN 3.0.0.3.0, including in its Service Advisor
- Alejandro Fernandez
On Feb. 9, 2017, 6:55 p.m., Vitalyi Brodetskyi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56511/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2017, 6:55 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
>
>
> Bugs: AMBARI-19947
> https://issues.apache.org/jira/browse/AMBARI-19947
>
>
> Repository: ambari
>
>
> Description
> -------
>
> We requires admin enables YARN preemption to make sure LLAP daemon can be launched in a shared cluster.
> And yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled=true need to be set to make preemption can happen under a fragmented cluster.
> So once admin enables preemption, we can automatically set yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled to true.
> This option (yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled) only supported by HDP-2.6 and after, it has no effect on HDP 2.5 and before.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml 555768e
> ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 957d625
>
> Diff: https://reviews.apache.org/r/56511/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Vitalyi Brodetskyi
>
>