You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Arun Suresh <as...@apache.org> on 2018/01/26 15:28:15 UTC
[VOTE] Merge YARN-6592 feature branch to trunk
Hello yarn-dev@
Based on the positive feedback from the DISCUSS thread [1], I'd like to
start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for
5 days, and will end Jan 31 7:30AM PDT.
This feature adds support for placing containers in YARN using rich
placement constraints. For example, this can be used by applications to
co-locate containers on a node or rack (*affinity *constraint), spread
containers across nodes or racks (*anti-affinity* constraint), or even
specify the maximum number of containers on a node/rack (*cardinality *
constraint).
We have integrated this feature into the Distributed-Shell application for
feature testing. We have performed end-to-end testing on moderately-sized
clusters to verify that constraints work fine. Performance tests have been
done via both SLS tests and Capacity Scheduler performance unit tests, and
no regression was found. We have opened a JIRA to track Jenkins acceptance
of the aggregated patch [3]. Documentation is in the process of being
completed [4]. You can also check our design document for more details [5].
Config flags are needed to enable this feature and it should not have any
effect on YARN when turned off. Once merged, we plan to work on further
improvements, which can be tracked in the umbrella YARN-7812 [6].
Kindly do take a look at the branch and raise issues/concerns that need to
be addressed before the merge.
Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
their contributions to this effort, as well as Subru, Chris, Carlo, and
Vinod for their inputs and discussions.
Cheers
-Arun
[1]
http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%2BfXnXhyNA%40mail.gmail.com%3E
[2] https://issues.apache.org/jira/browse/YARN-6592
[3] https://issues.apache.org/jira/browse/YARN-7792
[4] https://issues.apache.org/jira/browse/YARN-7780
[5] https://issues.apache.org/jira/secure/attachment/
12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
[6] https://issues.apache.org/jira/browse/YARN-7812
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Carlo Aldo Curino <ca...@gmail.com>.
+1
On Jan 26, 2018 9:28 AM, "Sunil G" <su...@apache.org> wrote:
> +1
>
> - Sunil
>
>
> On Fri, Jan 26, 2018 at 8:58 PM Arun Suresh <as...@apache.org> wrote:
>
>> Hello yarn-dev@
>>
>> Based on the positive feedback from the DISCUSS thread [1], I'd like to
>> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for
>> 5 days, and will end Jan 31 7:30AM PDT.
>>
>> This feature adds support for placing containers in YARN using rich
>> placement constraints. For example, this can be used by applications to
>> co-locate containers on a node or rack (*affinity *constraint), spread
>> containers across nodes or racks (*anti-affinity* constraint), or even
>> specify the maximum number of containers on a node/rack (*cardinality *
>> constraint).
>>
>> We have integrated this feature into the Distributed-Shell application
>> for feature testing. We have performed end-to-end testing on
>> moderately-sized clusters to verify that constraints work fine. Performance
>> tests have been done via both SLS tests and Capacity Scheduler performance
>> unit tests, and no regression was found. We have opened a JIRA to track
>> Jenkins acceptance of the aggregated patch [3]. Documentation is in the
>> process of being completed [4]. You can also check our design document for
>> more details [5].
>>
>> Config flags are needed to enable this feature and it should not have any
>> effect on YARN when turned off. Once merged, we plan to work on further
>> improvements, which can be tracked in the umbrella YARN-7812 [6].
>>
>> Kindly do take a look at the branch and raise issues/concerns that need
>> to be addressed before the merge.
>>
>> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
>> their contributions to this effort, as well as Subru, Chris, Carlo, and
>> Vinod for their inputs and discussions.
>>
>>
>> Cheers
>> -Arun
>>
>> [1] http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/
>> 201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%
>> 2BfXnXhyNA%40mail.gmail.com%3E
>> [2] https://issues.apache.org/jira/browse/YARN-6592
>> [3] https://issues.apache.org/jira/browse/YARN-7792
>> [4] https://issues.apache.org/jira/browse/YARN-7780
>> [5] https://issues.apache.org/jira/secure/attachment/
>> 12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
>> [6] https://issues.apache.org/jira/browse/YARN-7812
>>
>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Carlo Aldo Curino <ca...@gmail.com>.
+1
On Jan 26, 2018 9:28 AM, "Sunil G" <su...@apache.org> wrote:
> +1
>
> - Sunil
>
>
> On Fri, Jan 26, 2018 at 8:58 PM Arun Suresh <as...@apache.org> wrote:
>
>> Hello yarn-dev@
>>
>> Based on the positive feedback from the DISCUSS thread [1], I'd like to
>> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for
>> 5 days, and will end Jan 31 7:30AM PDT.
>>
>> This feature adds support for placing containers in YARN using rich
>> placement constraints. For example, this can be used by applications to
>> co-locate containers on a node or rack (*affinity *constraint), spread
>> containers across nodes or racks (*anti-affinity* constraint), or even
>> specify the maximum number of containers on a node/rack (*cardinality *
>> constraint).
>>
>> We have integrated this feature into the Distributed-Shell application
>> for feature testing. We have performed end-to-end testing on
>> moderately-sized clusters to verify that constraints work fine. Performance
>> tests have been done via both SLS tests and Capacity Scheduler performance
>> unit tests, and no regression was found. We have opened a JIRA to track
>> Jenkins acceptance of the aggregated patch [3]. Documentation is in the
>> process of being completed [4]. You can also check our design document for
>> more details [5].
>>
>> Config flags are needed to enable this feature and it should not have any
>> effect on YARN when turned off. Once merged, we plan to work on further
>> improvements, which can be tracked in the umbrella YARN-7812 [6].
>>
>> Kindly do take a look at the branch and raise issues/concerns that need
>> to be addressed before the merge.
>>
>> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
>> their contributions to this effort, as well as Subru, Chris, Carlo, and
>> Vinod for their inputs and discussions.
>>
>>
>> Cheers
>> -Arun
>>
>> [1] http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/
>> 201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%
>> 2BfXnXhyNA%40mail.gmail.com%3E
>> [2] https://issues.apache.org/jira/browse/YARN-6592
>> [3] https://issues.apache.org/jira/browse/YARN-7792
>> [4] https://issues.apache.org/jira/browse/YARN-7780
>> [5] https://issues.apache.org/jira/secure/attachment/
>> 12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
>> [6] https://issues.apache.org/jira/browse/YARN-7812
>>
>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Carlo Aldo Curino <ca...@gmail.com>.
+1
On Jan 26, 2018 9:28 AM, "Sunil G" <su...@apache.org> wrote:
> +1
>
> - Sunil
>
>
> On Fri, Jan 26, 2018 at 8:58 PM Arun Suresh <as...@apache.org> wrote:
>
>> Hello yarn-dev@
>>
>> Based on the positive feedback from the DISCUSS thread [1], I'd like to
>> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for
>> 5 days, and will end Jan 31 7:30AM PDT.
>>
>> This feature adds support for placing containers in YARN using rich
>> placement constraints. For example, this can be used by applications to
>> co-locate containers on a node or rack (*affinity *constraint), spread
>> containers across nodes or racks (*anti-affinity* constraint), or even
>> specify the maximum number of containers on a node/rack (*cardinality *
>> constraint).
>>
>> We have integrated this feature into the Distributed-Shell application
>> for feature testing. We have performed end-to-end testing on
>> moderately-sized clusters to verify that constraints work fine. Performance
>> tests have been done via both SLS tests and Capacity Scheduler performance
>> unit tests, and no regression was found. We have opened a JIRA to track
>> Jenkins acceptance of the aggregated patch [3]. Documentation is in the
>> process of being completed [4]. You can also check our design document for
>> more details [5].
>>
>> Config flags are needed to enable this feature and it should not have any
>> effect on YARN when turned off. Once merged, we plan to work on further
>> improvements, which can be tracked in the umbrella YARN-7812 [6].
>>
>> Kindly do take a look at the branch and raise issues/concerns that need
>> to be addressed before the merge.
>>
>> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
>> their contributions to this effort, as well as Subru, Chris, Carlo, and
>> Vinod for their inputs and discussions.
>>
>>
>> Cheers
>> -Arun
>>
>> [1] http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/
>> 201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%
>> 2BfXnXhyNA%40mail.gmail.com%3E
>> [2] https://issues.apache.org/jira/browse/YARN-6592
>> [3] https://issues.apache.org/jira/browse/YARN-7792
>> [4] https://issues.apache.org/jira/browse/YARN-7780
>> [5] https://issues.apache.org/jira/secure/attachment/
>> 12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
>> [6] https://issues.apache.org/jira/browse/YARN-7812
>>
>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Sunil G <su...@apache.org>.
+1
- Sunil
On Fri, Jan 26, 2018 at 8:58 PM Arun Suresh <as...@apache.org> wrote:
> Hello yarn-dev@
>
> Based on the positive feedback from the DISCUSS thread [1], I'd like to
> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for
> 5 days, and will end Jan 31 7:30AM PDT.
>
> This feature adds support for placing containers in YARN using rich
> placement constraints. For example, this can be used by applications to
> co-locate containers on a node or rack (*affinity *constraint), spread
> containers across nodes or racks (*anti-affinity* constraint), or even
> specify the maximum number of containers on a node/rack (*cardinality *
> constraint).
>
> We have integrated this feature into the Distributed-Shell application for
> feature testing. We have performed end-to-end testing on moderately-sized
> clusters to verify that constraints work fine. Performance tests have been
> done via both SLS tests and Capacity Scheduler performance unit tests, and
> no regression was found. We have opened a JIRA to track Jenkins acceptance
> of the aggregated patch [3]. Documentation is in the process of being
> completed [4]. You can also check our design document for more details [5].
>
> Config flags are needed to enable this feature and it should not have any
> effect on YARN when turned off. Once merged, we plan to work on further
> improvements, which can be tracked in the umbrella YARN-7812 [6].
>
> Kindly do take a look at the branch and raise issues/concerns that need to
> be addressed before the merge.
>
> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
> their contributions to this effort, as well as Subru, Chris, Carlo, and
> Vinod for their inputs and discussions.
>
>
> Cheers
> -Arun
>
> [1]
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> [2] https://issues.apache.org/jira/browse/YARN-6592
> [3] https://issues.apache.org/jira/browse/YARN-7792
> [4] https://issues.apache.org/jira/browse/YARN-7780
> [5]
> https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> [6] https://issues.apache.org/jira/browse/YARN-7812
>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Arun Suresh <as...@apache.org>.
Here's my +1 as well.
With 6 +1s and no -1s, the VOTE passes.
Thanks to everyone who participated.
Will be merging this shortly.
Cheers
-Arun
On Jan 26, 2018 5:46 PM, "Weiwei Yang" <ch...@hotmail.com> wrote:
> +1
> We are also looking forward to this : )
>
> --
> Weiwei
>
> On 27 Jan 2018, 9:16 AM +0800, Wangda Tan <wh...@gmail.com>, wrote:
> +1
>
> On Sat, Jan 27, 2018 at 2:05 AM, Chris Douglas <cdouglas@apache.org
> <ma...@apache.org>> wrote:
> +1 Looking forward to this. -C
>
> On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <asuresh@apache.org<mailto:
> asuresh@apache.org>> wrote:
> > Hello yarn-dev@
> >
> > Based on the positive feedback from the DISCUSS thread [1], I'd like to
> > start a formal vote to merge YARN-6592 [2] to trunk. The vote will run
> for 5
> > days, and will end Jan 31 7:30AM PDT.
> >
> > This feature adds support for placing containers in YARN using rich
> > placement constraints. For example, this can be used by applications to
> > co-locate containers on a node or rack (affinity constraint), spread
> > containers across nodes or racks (anti-affinity constraint), or even
> specify
> > the maximum number of containers on a node/rack (cardinality constraint).
> >
> > We have integrated this feature into the Distributed-Shell application
> for
> > feature testing. We have performed end-to-end testing on moderately-sized
> > clusters to verify that constraints work fine. Performance tests have
> been
> > done via both SLS tests and Capacity Scheduler performance unit tests,
> and
> > no regression was found. We have opened a JIRA to track Jenkins
> acceptance
> > of the aggregated patch [3]. Documentation is in the process of being
> > completed [4]. You can also check our design document for more details
> [5].
> >
> > Config flags are needed to enable this feature and it should not have any
> > effect on YARN when turned off. Once merged, we plan to work on further
> > improvements, which can be tracked in the umbrella YARN-7812 [6].
> >
> > Kindly do take a look at the branch and raise issues/concerns that need
> to
> > be addressed before the merge.
> >
> > Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
> their
> > contributions to this effort, as well as Subru, Chris, Carlo, and Vinod
> for
> > their inputs and discussions.
> >
> >
> > Cheers
> > -Arun
> >
> > [1]
> > http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201
> 801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_
> 4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> > [2] https://issues.apache.org/jira/browse/YARN-6592
> > [3] https://issues.apache.org/jira/browse/YARN-7792
> > [4] https://issues.apache.org/jira/browse/YARN-7780
> > [5]
> > https://issues.apache.org/jira/secure/attachment/12867869/
> YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> > [6] https://issues.apache.org/jira/browse/YARN-7812
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
> yarn-dev-unsubscribe@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
> yarn-dev-help@hadoop.apache.org>
>
>
>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Arun Suresh <as...@apache.org>.
Here's my +1 as well.
With 6 +1s and no -1s, the VOTE passes.
Thanks to everyone who participated.
Will be merging this shortly.
Cheers
-Arun
On Jan 26, 2018 5:46 PM, "Weiwei Yang" <ch...@hotmail.com> wrote:
> +1
> We are also looking forward to this : )
>
> --
> Weiwei
>
> On 27 Jan 2018, 9:16 AM +0800, Wangda Tan <wh...@gmail.com>, wrote:
> +1
>
> On Sat, Jan 27, 2018 at 2:05 AM, Chris Douglas <cdouglas@apache.org
> <ma...@apache.org>> wrote:
> +1 Looking forward to this. -C
>
> On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <asuresh@apache.org<mailto:
> asuresh@apache.org>> wrote:
> > Hello yarn-dev@
> >
> > Based on the positive feedback from the DISCUSS thread [1], I'd like to
> > start a formal vote to merge YARN-6592 [2] to trunk. The vote will run
> for 5
> > days, and will end Jan 31 7:30AM PDT.
> >
> > This feature adds support for placing containers in YARN using rich
> > placement constraints. For example, this can be used by applications to
> > co-locate containers on a node or rack (affinity constraint), spread
> > containers across nodes or racks (anti-affinity constraint), or even
> specify
> > the maximum number of containers on a node/rack (cardinality constraint).
> >
> > We have integrated this feature into the Distributed-Shell application
> for
> > feature testing. We have performed end-to-end testing on moderately-sized
> > clusters to verify that constraints work fine. Performance tests have
> been
> > done via both SLS tests and Capacity Scheduler performance unit tests,
> and
> > no regression was found. We have opened a JIRA to track Jenkins
> acceptance
> > of the aggregated patch [3]. Documentation is in the process of being
> > completed [4]. You can also check our design document for more details
> [5].
> >
> > Config flags are needed to enable this feature and it should not have any
> > effect on YARN when turned off. Once merged, we plan to work on further
> > improvements, which can be tracked in the umbrella YARN-7812 [6].
> >
> > Kindly do take a look at the branch and raise issues/concerns that need
> to
> > be addressed before the merge.
> >
> > Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
> their
> > contributions to this effort, as well as Subru, Chris, Carlo, and Vinod
> for
> > their inputs and discussions.
> >
> >
> > Cheers
> > -Arun
> >
> > [1]
> > http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201
> 801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_
> 4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> > [2] https://issues.apache.org/jira/browse/YARN-6592
> > [3] https://issues.apache.org/jira/browse/YARN-7792
> > [4] https://issues.apache.org/jira/browse/YARN-7780
> > [5]
> > https://issues.apache.org/jira/secure/attachment/12867869/
> YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> > [6] https://issues.apache.org/jira/browse/YARN-7812
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
> yarn-dev-unsubscribe@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
> yarn-dev-help@hadoop.apache.org>
>
>
>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Arun Suresh <as...@apache.org>.
Here's my +1 as well.
With 6 +1s and no -1s, the VOTE passes.
Thanks to everyone who participated.
Will be merging this shortly.
Cheers
-Arun
On Jan 26, 2018 5:46 PM, "Weiwei Yang" <ch...@hotmail.com> wrote:
> +1
> We are also looking forward to this : )
>
> --
> Weiwei
>
> On 27 Jan 2018, 9:16 AM +0800, Wangda Tan <wh...@gmail.com>, wrote:
> +1
>
> On Sat, Jan 27, 2018 at 2:05 AM, Chris Douglas <cdouglas@apache.org
> <ma...@apache.org>> wrote:
> +1 Looking forward to this. -C
>
> On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <asuresh@apache.org<mailto:
> asuresh@apache.org>> wrote:
> > Hello yarn-dev@
> >
> > Based on the positive feedback from the DISCUSS thread [1], I'd like to
> > start a formal vote to merge YARN-6592 [2] to trunk. The vote will run
> for 5
> > days, and will end Jan 31 7:30AM PDT.
> >
> > This feature adds support for placing containers in YARN using rich
> > placement constraints. For example, this can be used by applications to
> > co-locate containers on a node or rack (affinity constraint), spread
> > containers across nodes or racks (anti-affinity constraint), or even
> specify
> > the maximum number of containers on a node/rack (cardinality constraint).
> >
> > We have integrated this feature into the Distributed-Shell application
> for
> > feature testing. We have performed end-to-end testing on moderately-sized
> > clusters to verify that constraints work fine. Performance tests have
> been
> > done via both SLS tests and Capacity Scheduler performance unit tests,
> and
> > no regression was found. We have opened a JIRA to track Jenkins
> acceptance
> > of the aggregated patch [3]. Documentation is in the process of being
> > completed [4]. You can also check our design document for more details
> [5].
> >
> > Config flags are needed to enable this feature and it should not have any
> > effect on YARN when turned off. Once merged, we plan to work on further
> > improvements, which can be tracked in the umbrella YARN-7812 [6].
> >
> > Kindly do take a look at the branch and raise issues/concerns that need
> to
> > be addressed before the merge.
> >
> > Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
> their
> > contributions to this effort, as well as Subru, Chris, Carlo, and Vinod
> for
> > their inputs and discussions.
> >
> >
> > Cheers
> > -Arun
> >
> > [1]
> > http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201
> 801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_
> 4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> > [2] https://issues.apache.org/jira/browse/YARN-6592
> > [3] https://issues.apache.org/jira/browse/YARN-7792
> > [4] https://issues.apache.org/jira/browse/YARN-7780
> > [5]
> > https://issues.apache.org/jira/secure/attachment/12867869/
> YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> > [6] https://issues.apache.org/jira/browse/YARN-7812
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
> yarn-dev-unsubscribe@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
> yarn-dev-help@hadoop.apache.org>
>
>
>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Weiwei Yang <ch...@hotmail.com>.
+1
We are also looking forward to this : )
--
Weiwei
On 27 Jan 2018, 9:16 AM +0800, Wangda Tan <wh...@gmail.com>, wrote:
+1
On Sat, Jan 27, 2018 at 2:05 AM, Chris Douglas <cd...@apache.org>> wrote:
+1 Looking forward to this. -C
On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <as...@apache.org>> wrote:
> Hello yarn-dev@
>
> Based on the positive feedback from the DISCUSS thread [1], I'd like to
> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for 5
> days, and will end Jan 31 7:30AM PDT.
>
> This feature adds support for placing containers in YARN using rich
> placement constraints. For example, this can be used by applications to
> co-locate containers on a node or rack (affinity constraint), spread
> containers across nodes or racks (anti-affinity constraint), or even specify
> the maximum number of containers on a node/rack (cardinality constraint).
>
> We have integrated this feature into the Distributed-Shell application for
> feature testing. We have performed end-to-end testing on moderately-sized
> clusters to verify that constraints work fine. Performance tests have been
> done via both SLS tests and Capacity Scheduler performance unit tests, and
> no regression was found. We have opened a JIRA to track Jenkins acceptance
> of the aggregated patch [3]. Documentation is in the process of being
> completed [4]. You can also check our design document for more details [5].
>
> Config flags are needed to enable this feature and it should not have any
> effect on YARN when turned off. Once merged, we plan to work on further
> improvements, which can be tracked in the umbrella YARN-7812 [6].
>
> Kindly do take a look at the branch and raise issues/concerns that need to
> be addressed before the merge.
>
> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for their
> contributions to this effort, as well as Subru, Chris, Carlo, and Vinod for
> their inputs and discussions.
>
>
> Cheers
> -Arun
>
> [1]
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> [2] https://issues.apache.org/jira/browse/YARN-6592
> [3] https://issues.apache.org/jira/browse/YARN-7792
> [4] https://issues.apache.org/jira/browse/YARN-7780
> [5]
> https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> [6] https://issues.apache.org/jira/browse/YARN-7812
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Weiwei Yang <ch...@hotmail.com>.
+1
We are also looking forward to this : )
--
Weiwei
On 27 Jan 2018, 9:16 AM +0800, Wangda Tan <wh...@gmail.com>, wrote:
+1
On Sat, Jan 27, 2018 at 2:05 AM, Chris Douglas <cd...@apache.org>> wrote:
+1 Looking forward to this. -C
On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <as...@apache.org>> wrote:
> Hello yarn-dev@
>
> Based on the positive feedback from the DISCUSS thread [1], I'd like to
> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for 5
> days, and will end Jan 31 7:30AM PDT.
>
> This feature adds support for placing containers in YARN using rich
> placement constraints. For example, this can be used by applications to
> co-locate containers on a node or rack (affinity constraint), spread
> containers across nodes or racks (anti-affinity constraint), or even specify
> the maximum number of containers on a node/rack (cardinality constraint).
>
> We have integrated this feature into the Distributed-Shell application for
> feature testing. We have performed end-to-end testing on moderately-sized
> clusters to verify that constraints work fine. Performance tests have been
> done via both SLS tests and Capacity Scheduler performance unit tests, and
> no regression was found. We have opened a JIRA to track Jenkins acceptance
> of the aggregated patch [3]. Documentation is in the process of being
> completed [4]. You can also check our design document for more details [5].
>
> Config flags are needed to enable this feature and it should not have any
> effect on YARN when turned off. Once merged, we plan to work on further
> improvements, which can be tracked in the umbrella YARN-7812 [6].
>
> Kindly do take a look at the branch and raise issues/concerns that need to
> be addressed before the merge.
>
> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for their
> contributions to this effort, as well as Subru, Chris, Carlo, and Vinod for
> their inputs and discussions.
>
>
> Cheers
> -Arun
>
> [1]
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> [2] https://issues.apache.org/jira/browse/YARN-6592
> [3] https://issues.apache.org/jira/browse/YARN-7792
> [4] https://issues.apache.org/jira/browse/YARN-7780
> [5]
> https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> [6] https://issues.apache.org/jira/browse/YARN-7812
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Weiwei Yang <ch...@hotmail.com>.
+1
We are also looking forward to this : )
--
Weiwei
On 27 Jan 2018, 9:16 AM +0800, Wangda Tan <wh...@gmail.com>, wrote:
+1
On Sat, Jan 27, 2018 at 2:05 AM, Chris Douglas <cd...@apache.org>> wrote:
+1 Looking forward to this. -C
On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <as...@apache.org>> wrote:
> Hello yarn-dev@
>
> Based on the positive feedback from the DISCUSS thread [1], I'd like to
> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for 5
> days, and will end Jan 31 7:30AM PDT.
>
> This feature adds support for placing containers in YARN using rich
> placement constraints. For example, this can be used by applications to
> co-locate containers on a node or rack (affinity constraint), spread
> containers across nodes or racks (anti-affinity constraint), or even specify
> the maximum number of containers on a node/rack (cardinality constraint).
>
> We have integrated this feature into the Distributed-Shell application for
> feature testing. We have performed end-to-end testing on moderately-sized
> clusters to verify that constraints work fine. Performance tests have been
> done via both SLS tests and Capacity Scheduler performance unit tests, and
> no regression was found. We have opened a JIRA to track Jenkins acceptance
> of the aggregated patch [3]. Documentation is in the process of being
> completed [4]. You can also check our design document for more details [5].
>
> Config flags are needed to enable this feature and it should not have any
> effect on YARN when turned off. Once merged, we plan to work on further
> improvements, which can be tracked in the umbrella YARN-7812 [6].
>
> Kindly do take a look at the branch and raise issues/concerns that need to
> be addressed before the merge.
>
> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for their
> contributions to this effort, as well as Subru, Chris, Carlo, and Vinod for
> their inputs and discussions.
>
>
> Cheers
> -Arun
>
> [1]
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> [2] https://issues.apache.org/jira/browse/YARN-6592
> [3] https://issues.apache.org/jira/browse/YARN-7792
> [4] https://issues.apache.org/jira/browse/YARN-7780
> [5]
> https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> [6] https://issues.apache.org/jira/browse/YARN-7812
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Wangda Tan <wh...@gmail.com>.
+1
On Sat, Jan 27, 2018 at 2:05 AM, Chris Douglas <cd...@apache.org> wrote:
> +1 Looking forward to this. -C
>
> On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <as...@apache.org> wrote:
> > Hello yarn-dev@
> >
> > Based on the positive feedback from the DISCUSS thread [1], I'd like to
> > start a formal vote to merge YARN-6592 [2] to trunk. The vote will run
> for 5
> > days, and will end Jan 31 7:30AM PDT.
> >
> > This feature adds support for placing containers in YARN using rich
> > placement constraints. For example, this can be used by applications to
> > co-locate containers on a node or rack (affinity constraint), spread
> > containers across nodes or racks (anti-affinity constraint), or even
> specify
> > the maximum number of containers on a node/rack (cardinality constraint).
> >
> > We have integrated this feature into the Distributed-Shell application
> for
> > feature testing. We have performed end-to-end testing on moderately-sized
> > clusters to verify that constraints work fine. Performance tests have
> been
> > done via both SLS tests and Capacity Scheduler performance unit tests,
> and
> > no regression was found. We have opened a JIRA to track Jenkins
> acceptance
> > of the aggregated patch [3]. Documentation is in the process of being
> > completed [4]. You can also check our design document for more details
> [5].
> >
> > Config flags are needed to enable this feature and it should not have any
> > effect on YARN when turned off. Once merged, we plan to work on further
> > improvements, which can be tracked in the umbrella YARN-7812 [6].
> >
> > Kindly do take a look at the branch and raise issues/concerns that need
> to
> > be addressed before the merge.
> >
> > Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
> their
> > contributions to this effort, as well as Subru, Chris, Carlo, and Vinod
> for
> > their inputs and discussions.
> >
> >
> > Cheers
> > -Arun
> >
> > [1]
> > http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/
> 201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%
> 2BfXnXhyNA%40mail.gmail.com%3E
> > [2] https://issues.apache.org/jira/browse/YARN-6592
> > [3] https://issues.apache.org/jira/browse/YARN-7792
> > [4] https://issues.apache.org/jira/browse/YARN-7780
> > [5]
> > https://issues.apache.org/jira/secure/attachment/
> 12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> > [6] https://issues.apache.org/jira/browse/YARN-7812
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Wangda Tan <wh...@gmail.com>.
+1
On Sat, Jan 27, 2018 at 2:05 AM, Chris Douglas <cd...@apache.org> wrote:
> +1 Looking forward to this. -C
>
> On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <as...@apache.org> wrote:
> > Hello yarn-dev@
> >
> > Based on the positive feedback from the DISCUSS thread [1], I'd like to
> > start a formal vote to merge YARN-6592 [2] to trunk. The vote will run
> for 5
> > days, and will end Jan 31 7:30AM PDT.
> >
> > This feature adds support for placing containers in YARN using rich
> > placement constraints. For example, this can be used by applications to
> > co-locate containers on a node or rack (affinity constraint), spread
> > containers across nodes or racks (anti-affinity constraint), or even
> specify
> > the maximum number of containers on a node/rack (cardinality constraint).
> >
> > We have integrated this feature into the Distributed-Shell application
> for
> > feature testing. We have performed end-to-end testing on moderately-sized
> > clusters to verify that constraints work fine. Performance tests have
> been
> > done via both SLS tests and Capacity Scheduler performance unit tests,
> and
> > no regression was found. We have opened a JIRA to track Jenkins
> acceptance
> > of the aggregated patch [3]. Documentation is in the process of being
> > completed [4]. You can also check our design document for more details
> [5].
> >
> > Config flags are needed to enable this feature and it should not have any
> > effect on YARN when turned off. Once merged, we plan to work on further
> > improvements, which can be tracked in the umbrella YARN-7812 [6].
> >
> > Kindly do take a look at the branch and raise issues/concerns that need
> to
> > be addressed before the merge.
> >
> > Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
> their
> > contributions to this effort, as well as Subru, Chris, Carlo, and Vinod
> for
> > their inputs and discussions.
> >
> >
> > Cheers
> > -Arun
> >
> > [1]
> > http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/
> 201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%
> 2BfXnXhyNA%40mail.gmail.com%3E
> > [2] https://issues.apache.org/jira/browse/YARN-6592
> > [3] https://issues.apache.org/jira/browse/YARN-7792
> > [4] https://issues.apache.org/jira/browse/YARN-7780
> > [5]
> > https://issues.apache.org/jira/secure/attachment/
> 12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> > [6] https://issues.apache.org/jira/browse/YARN-7812
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Wangda Tan <wh...@gmail.com>.
+1
On Sat, Jan 27, 2018 at 2:05 AM, Chris Douglas <cd...@apache.org> wrote:
> +1 Looking forward to this. -C
>
> On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <as...@apache.org> wrote:
> > Hello yarn-dev@
> >
> > Based on the positive feedback from the DISCUSS thread [1], I'd like to
> > start a formal vote to merge YARN-6592 [2] to trunk. The vote will run
> for 5
> > days, and will end Jan 31 7:30AM PDT.
> >
> > This feature adds support for placing containers in YARN using rich
> > placement constraints. For example, this can be used by applications to
> > co-locate containers on a node or rack (affinity constraint), spread
> > containers across nodes or racks (anti-affinity constraint), or even
> specify
> > the maximum number of containers on a node/rack (cardinality constraint).
> >
> > We have integrated this feature into the Distributed-Shell application
> for
> > feature testing. We have performed end-to-end testing on moderately-sized
> > clusters to verify that constraints work fine. Performance tests have
> been
> > done via both SLS tests and Capacity Scheduler performance unit tests,
> and
> > no regression was found. We have opened a JIRA to track Jenkins
> acceptance
> > of the aggregated patch [3]. Documentation is in the process of being
> > completed [4]. You can also check our design document for more details
> [5].
> >
> > Config flags are needed to enable this feature and it should not have any
> > effect on YARN when turned off. Once merged, we plan to work on further
> > improvements, which can be tracked in the umbrella YARN-7812 [6].
> >
> > Kindly do take a look at the branch and raise issues/concerns that need
> to
> > be addressed before the merge.
> >
> > Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
> their
> > contributions to this effort, as well as Subru, Chris, Carlo, and Vinod
> for
> > their inputs and discussions.
> >
> >
> > Cheers
> > -Arun
> >
> > [1]
> > http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/
> 201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%
> 2BfXnXhyNA%40mail.gmail.com%3E
> > [2] https://issues.apache.org/jira/browse/YARN-6592
> > [3] https://issues.apache.org/jira/browse/YARN-7792
> > [4] https://issues.apache.org/jira/browse/YARN-7780
> > [5]
> > https://issues.apache.org/jira/secure/attachment/
> 12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> > [6] https://issues.apache.org/jira/browse/YARN-7812
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Chris Douglas <cd...@apache.org>.
+1 Looking forward to this. -C
On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <as...@apache.org> wrote:
> Hello yarn-dev@
>
> Based on the positive feedback from the DISCUSS thread [1], I'd like to
> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for 5
> days, and will end Jan 31 7:30AM PDT.
>
> This feature adds support for placing containers in YARN using rich
> placement constraints. For example, this can be used by applications to
> co-locate containers on a node or rack (affinity constraint), spread
> containers across nodes or racks (anti-affinity constraint), or even specify
> the maximum number of containers on a node/rack (cardinality constraint).
>
> We have integrated this feature into the Distributed-Shell application for
> feature testing. We have performed end-to-end testing on moderately-sized
> clusters to verify that constraints work fine. Performance tests have been
> done via both SLS tests and Capacity Scheduler performance unit tests, and
> no regression was found. We have opened a JIRA to track Jenkins acceptance
> of the aggregated patch [3]. Documentation is in the process of being
> completed [4]. You can also check our design document for more details [5].
>
> Config flags are needed to enable this feature and it should not have any
> effect on YARN when turned off. Once merged, we plan to work on further
> improvements, which can be tracked in the umbrella YARN-7812 [6].
>
> Kindly do take a look at the branch and raise issues/concerns that need to
> be addressed before the merge.
>
> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for their
> contributions to this effort, as well as Subru, Chris, Carlo, and Vinod for
> their inputs and discussions.
>
>
> Cheers
> -Arun
>
> [1]
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> [2] https://issues.apache.org/jira/browse/YARN-6592
> [3] https://issues.apache.org/jira/browse/YARN-7792
> [4] https://issues.apache.org/jira/browse/YARN-7780
> [5]
> https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> [6] https://issues.apache.org/jira/browse/YARN-7812
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Chris Douglas <cd...@apache.org>.
+1 Looking forward to this. -C
On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <as...@apache.org> wrote:
> Hello yarn-dev@
>
> Based on the positive feedback from the DISCUSS thread [1], I'd like to
> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for 5
> days, and will end Jan 31 7:30AM PDT.
>
> This feature adds support for placing containers in YARN using rich
> placement constraints. For example, this can be used by applications to
> co-locate containers on a node or rack (affinity constraint), spread
> containers across nodes or racks (anti-affinity constraint), or even specify
> the maximum number of containers on a node/rack (cardinality constraint).
>
> We have integrated this feature into the Distributed-Shell application for
> feature testing. We have performed end-to-end testing on moderately-sized
> clusters to verify that constraints work fine. Performance tests have been
> done via both SLS tests and Capacity Scheduler performance unit tests, and
> no regression was found. We have opened a JIRA to track Jenkins acceptance
> of the aggregated patch [3]. Documentation is in the process of being
> completed [4]. You can also check our design document for more details [5].
>
> Config flags are needed to enable this feature and it should not have any
> effect on YARN when turned off. Once merged, we plan to work on further
> improvements, which can be tracked in the umbrella YARN-7812 [6].
>
> Kindly do take a look at the branch and raise issues/concerns that need to
> be addressed before the merge.
>
> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for their
> contributions to this effort, as well as Subru, Chris, Carlo, and Vinod for
> their inputs and discussions.
>
>
> Cheers
> -Arun
>
> [1]
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> [2] https://issues.apache.org/jira/browse/YARN-6592
> [3] https://issues.apache.org/jira/browse/YARN-7792
> [4] https://issues.apache.org/jira/browse/YARN-7780
> [5]
> https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> [6] https://issues.apache.org/jira/browse/YARN-7812
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Sunil G <su...@apache.org>.
+1
- Sunil
On Fri, Jan 26, 2018 at 8:58 PM Arun Suresh <as...@apache.org> wrote:
> Hello yarn-dev@
>
> Based on the positive feedback from the DISCUSS thread [1], I'd like to
> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for
> 5 days, and will end Jan 31 7:30AM PDT.
>
> This feature adds support for placing containers in YARN using rich
> placement constraints. For example, this can be used by applications to
> co-locate containers on a node or rack (*affinity *constraint), spread
> containers across nodes or racks (*anti-affinity* constraint), or even
> specify the maximum number of containers on a node/rack (*cardinality *
> constraint).
>
> We have integrated this feature into the Distributed-Shell application for
> feature testing. We have performed end-to-end testing on moderately-sized
> clusters to verify that constraints work fine. Performance tests have been
> done via both SLS tests and Capacity Scheduler performance unit tests, and
> no regression was found. We have opened a JIRA to track Jenkins acceptance
> of the aggregated patch [3]. Documentation is in the process of being
> completed [4]. You can also check our design document for more details [5].
>
> Config flags are needed to enable this feature and it should not have any
> effect on YARN when turned off. Once merged, we plan to work on further
> improvements, which can be tracked in the umbrella YARN-7812 [6].
>
> Kindly do take a look at the branch and raise issues/concerns that need to
> be addressed before the merge.
>
> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
> their contributions to this effort, as well as Subru, Chris, Carlo, and
> Vinod for their inputs and discussions.
>
>
> Cheers
> -Arun
>
> [1]
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> [2] https://issues.apache.org/jira/browse/YARN-6592
> [3] https://issues.apache.org/jira/browse/YARN-7792
> [4] https://issues.apache.org/jira/browse/YARN-7780
> [5]
> https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> [6] https://issues.apache.org/jira/browse/YARN-7812
>
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Chris Douglas <cd...@apache.org>.
+1 Looking forward to this. -C
On Fri, Jan 26, 2018 at 7:28 AM, Arun Suresh <as...@apache.org> wrote:
> Hello yarn-dev@
>
> Based on the positive feedback from the DISCUSS thread [1], I'd like to
> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for 5
> days, and will end Jan 31 7:30AM PDT.
>
> This feature adds support for placing containers in YARN using rich
> placement constraints. For example, this can be used by applications to
> co-locate containers on a node or rack (affinity constraint), spread
> containers across nodes or racks (anti-affinity constraint), or even specify
> the maximum number of containers on a node/rack (cardinality constraint).
>
> We have integrated this feature into the Distributed-Shell application for
> feature testing. We have performed end-to-end testing on moderately-sized
> clusters to verify that constraints work fine. Performance tests have been
> done via both SLS tests and Capacity Scheduler performance unit tests, and
> no regression was found. We have opened a JIRA to track Jenkins acceptance
> of the aggregated patch [3]. Documentation is in the process of being
> completed [4]. You can also check our design document for more details [5].
>
> Config flags are needed to enable this feature and it should not have any
> effect on YARN when turned off. Once merged, we plan to work on further
> improvements, which can be tracked in the umbrella YARN-7812 [6].
>
> Kindly do take a look at the branch and raise issues/concerns that need to
> be addressed before the merge.
>
> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for their
> contributions to this effort, as well as Subru, Chris, Carlo, and Vinod for
> their inputs and discussions.
>
>
> Cheers
> -Arun
>
> [1]
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> [2] https://issues.apache.org/jira/browse/YARN-6592
> [3] https://issues.apache.org/jira/browse/YARN-7792
> [4] https://issues.apache.org/jira/browse/YARN-7780
> [5]
> https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> [6] https://issues.apache.org/jira/browse/YARN-7812
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Merge YARN-6592 feature branch to trunk
Posted by Sunil G <su...@apache.org>.
+1
- Sunil
On Fri, Jan 26, 2018 at 8:58 PM Arun Suresh <as...@apache.org> wrote:
> Hello yarn-dev@
>
> Based on the positive feedback from the DISCUSS thread [1], I'd like to
> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for
> 5 days, and will end Jan 31 7:30AM PDT.
>
> This feature adds support for placing containers in YARN using rich
> placement constraints. For example, this can be used by applications to
> co-locate containers on a node or rack (*affinity *constraint), spread
> containers across nodes or racks (*anti-affinity* constraint), or even
> specify the maximum number of containers on a node/rack (*cardinality *
> constraint).
>
> We have integrated this feature into the Distributed-Shell application for
> feature testing. We have performed end-to-end testing on moderately-sized
> clusters to verify that constraints work fine. Performance tests have been
> done via both SLS tests and Capacity Scheduler performance unit tests, and
> no regression was found. We have opened a JIRA to track Jenkins acceptance
> of the aggregated patch [3]. Documentation is in the process of being
> completed [4]. You can also check our design document for more details [5].
>
> Config flags are needed to enable this feature and it should not have any
> effect on YARN when turned off. Once merged, we plan to work on further
> improvements, which can be tracked in the umbrella YARN-7812 [6].
>
> Kindly do take a look at the branch and raise issues/concerns that need to
> be addressed before the merge.
>
> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
> their contributions to this effort, as well as Subru, Chris, Carlo, and
> Vinod for their inputs and discussions.
>
>
> Cheers
> -Arun
>
> [1]
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%2BfXnXhyNA%40mail.gmail.com%3E
> [2] https://issues.apache.org/jira/browse/YARN-6592
> [3] https://issues.apache.org/jira/browse/YARN-7792
> [4] https://issues.apache.org/jira/browse/YARN-7780
> [5]
> https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> [6] https://issues.apache.org/jira/browse/YARN-7812
>