You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Karthik Kambatla <ka...@cloudera.com> on 2016/11/17 05:17:10 UTC
[VOTE] Merge feature branch YARN-4752. FairScheduler preemption overhaul
Hi folks,
Based on the positive feedback on the DISCUSS thread [1], I would like to
start a formal vote to merge feature branch YARN-4752 to trunk. The
cumulative patch is posted to YARN-4752, and squashed into one commit here
[2].
The changes include:
1. Preemption considers individual ResourceRequests to satisfy.
(YARN-5605)
2. Preemption now works within a leaf queue and across sibling leaf
queues. (YARN-5605)
3. Comprehensive unit tests for app starvation and preemption - minshare
and fairshare. (YARN-5783 and YARN-5819)
4. Follow up JIRAs for further improvements, also annotated in the code.
(YARN-5885). Will continue working on these improvements post merge.
These changes are strictly internal to the scheduler, mostly FairScheduler.
The branch followed RTC.
Thanks to Arun for the initial design recommendations, Daniel for his
diligent reviews of sub-tasks, Arun and Sangjin for a high-level review of
the branch in its entirety.
The vote will run for 7 days, and will end on Nov 23.
Thanks
Karthik
[1]
http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201611.mbox/%3CCALwhT97kAO6zuUjXTsmHgw_j-q-M9t9Gm9Vf11aogD%2B%2Bqgp2jQ%40mail.gmail.com%3E
[2]
https://github.com/apache/hadoop/commit/386f3a2af749bc96bdca1dc2302a3589773091dc
Re: [VOTE] Merge feature branch YARN-4752. FairScheduler preemption overhaul
Posted by Karthik Kambatla <ka...@cloudera.com>.
Let me conclude the vote with my +1.
With 4 +1s and no -1s, the vote passes. Thanks Daniel, Arun and Sangjin for
voting. Given the +1 from Jenkins as well, I ll go ahead and merge the
branch as a single commit.
Thanks
Karthik
On Thu, Nov 17, 2016 at 10:01 AM, Sangjin Lee <sj...@apache.org> wrote:
> +1
>
> On Thu, Nov 17, 2016 at 8:21 AM, Arun Suresh <as...@apache.org> wrote:
>
>> +1
>>
>>
>> On Thu, Nov 17, 2016 at 6:21 AM, Daniel Templeton <da...@cloudera.com>
>> wrote:
>>
>> > +1 on the merge. Nice work, Karthik!
>> >
>> > Daniel
>> >
>> >
>> > On 11/16/16 9:17 PM, Karthik Kambatla wrote:
>> >
>> >> Hi folks,
>> >>
>> >> Based on the positive feedback on the DISCUSS thread [1], I would like
>> to
>> >> start a formal vote to merge feature branch YARN-4752 to trunk. The
>> >> cumulative patch is posted to YARN-4752, and squashed into one commit
>> here
>> >> [2].
>> >>
>> >> The changes include:
>> >>
>> >> 1. Preemption considers individual ResourceRequests to satisfy.
>> >> (YARN-5605)
>> >> 2. Preemption now works within a leaf queue and across sibling leaf
>> >> queues. (YARN-5605)
>> >> 3. Comprehensive unit tests for app starvation and preemption -
>> >> minshare
>> >> and fairshare. (YARN-5783 and YARN-5819)
>> >> 4. Follow up JIRAs for further improvements, also annotated in the
>> >> code.
>> >> (YARN-5885). Will continue working on these improvements post
>> merge.
>> >>
>> >> These changes are strictly internal to the scheduler, mostly
>> >> FairScheduler.
>> >> The branch followed RTC.
>> >>
>> >> Thanks to Arun for the initial design recommendations, Daniel for his
>> >> diligent reviews of sub-tasks, Arun and Sangjin for a high-level
>> review of
>> >> the branch in its entirety.
>> >>
>> >> The vote will run for 7 days, and will end on Nov 23.
>> >>
>> >> Thanks
>> >> Karthik
>> >>
>> >> [1]
>> >> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201
>> >> 611.mbox/%3CCALwhT97kAO6zuUjXTsmHgw_j-q-M9t9Gm9Vf11aogD%2B%
>> >> 2Bqgp2jQ%40mail.gmail.com%3E
>> >> [2]
>> >> https://github.com/apache/hadoop/commit/386f3a2af749bc96bdca
>> >> 1dc2302a3589773091dc
>> >>
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>> >
>> >
>>
>
>
Re: [VOTE] Merge feature branch YARN-4752. FairScheduler preemption overhaul
Posted by Sangjin Lee <sj...@apache.org>.
+1
On Thu, Nov 17, 2016 at 8:21 AM, Arun Suresh <as...@apache.org> wrote:
> +1
>
>
> On Thu, Nov 17, 2016 at 6:21 AM, Daniel Templeton <da...@cloudera.com>
> wrote:
>
> > +1 on the merge. Nice work, Karthik!
> >
> > Daniel
> >
> >
> > On 11/16/16 9:17 PM, Karthik Kambatla wrote:
> >
> >> Hi folks,
> >>
> >> Based on the positive feedback on the DISCUSS thread [1], I would like
> to
> >> start a formal vote to merge feature branch YARN-4752 to trunk. The
> >> cumulative patch is posted to YARN-4752, and squashed into one commit
> here
> >> [2].
> >>
> >> The changes include:
> >>
> >> 1. Preemption considers individual ResourceRequests to satisfy.
> >> (YARN-5605)
> >> 2. Preemption now works within a leaf queue and across sibling leaf
> >> queues. (YARN-5605)
> >> 3. Comprehensive unit tests for app starvation and preemption -
> >> minshare
> >> and fairshare. (YARN-5783 and YARN-5819)
> >> 4. Follow up JIRAs for further improvements, also annotated in the
> >> code.
> >> (YARN-5885). Will continue working on these improvements post merge.
> >>
> >> These changes are strictly internal to the scheduler, mostly
> >> FairScheduler.
> >> The branch followed RTC.
> >>
> >> Thanks to Arun for the initial design recommendations, Daniel for his
> >> diligent reviews of sub-tasks, Arun and Sangjin for a high-level review
> of
> >> the branch in its entirety.
> >>
> >> The vote will run for 7 days, and will end on Nov 23.
> >>
> >> Thanks
> >> Karthik
> >>
> >> [1]
> >> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201
> >> 611.mbox/%3CCALwhT97kAO6zuUjXTsmHgw_j-q-M9t9Gm9Vf11aogD%2B%
> >> 2Bqgp2jQ%40mail.gmail.com%3E
> >> [2]
> >> https://github.com/apache/hadoop/commit/386f3a2af749bc96bdca
> >> 1dc2302a3589773091dc
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
>
Re: [VOTE] Merge feature branch YARN-4752. FairScheduler preemption overhaul
Posted by Arun Suresh <as...@apache.org>.
+1
On Thu, Nov 17, 2016 at 6:21 AM, Daniel Templeton <da...@cloudera.com>
wrote:
> +1 on the merge. Nice work, Karthik!
>
> Daniel
>
>
> On 11/16/16 9:17 PM, Karthik Kambatla wrote:
>
>> Hi folks,
>>
>> Based on the positive feedback on the DISCUSS thread [1], I would like to
>> start a formal vote to merge feature branch YARN-4752 to trunk. The
>> cumulative patch is posted to YARN-4752, and squashed into one commit here
>> [2].
>>
>> The changes include:
>>
>> 1. Preemption considers individual ResourceRequests to satisfy.
>> (YARN-5605)
>> 2. Preemption now works within a leaf queue and across sibling leaf
>> queues. (YARN-5605)
>> 3. Comprehensive unit tests for app starvation and preemption -
>> minshare
>> and fairshare. (YARN-5783 and YARN-5819)
>> 4. Follow up JIRAs for further improvements, also annotated in the
>> code.
>> (YARN-5885). Will continue working on these improvements post merge.
>>
>> These changes are strictly internal to the scheduler, mostly
>> FairScheduler.
>> The branch followed RTC.
>>
>> Thanks to Arun for the initial design recommendations, Daniel for his
>> diligent reviews of sub-tasks, Arun and Sangjin for a high-level review of
>> the branch in its entirety.
>>
>> The vote will run for 7 days, and will end on Nov 23.
>>
>> Thanks
>> Karthik
>>
>> [1]
>> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201
>> 611.mbox/%3CCALwhT97kAO6zuUjXTsmHgw_j-q-M9t9Gm9Vf11aogD%2B%
>> 2Bqgp2jQ%40mail.gmail.com%3E
>> [2]
>> https://github.com/apache/hadoop/commit/386f3a2af749bc96bdca
>> 1dc2302a3589773091dc
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [VOTE] Merge feature branch YARN-4752. FairScheduler preemption
overhaul
Posted by Daniel Templeton <da...@cloudera.com>.
+1 on the merge. Nice work, Karthik!
Daniel
On 11/16/16 9:17 PM, Karthik Kambatla wrote:
> Hi folks,
>
> Based on the positive feedback on the DISCUSS thread [1], I would like to
> start a formal vote to merge feature branch YARN-4752 to trunk. The
> cumulative patch is posted to YARN-4752, and squashed into one commit here
> [2].
>
> The changes include:
>
> 1. Preemption considers individual ResourceRequests to satisfy.
> (YARN-5605)
> 2. Preemption now works within a leaf queue and across sibling leaf
> queues. (YARN-5605)
> 3. Comprehensive unit tests for app starvation and preemption - minshare
> and fairshare. (YARN-5783 and YARN-5819)
> 4. Follow up JIRAs for further improvements, also annotated in the code.
> (YARN-5885). Will continue working on these improvements post merge.
>
> These changes are strictly internal to the scheduler, mostly FairScheduler.
> The branch followed RTC.
>
> Thanks to Arun for the initial design recommendations, Daniel for his
> diligent reviews of sub-tasks, Arun and Sangjin for a high-level review of
> the branch in its entirety.
>
> The vote will run for 7 days, and will end on Nov 23.
>
> Thanks
> Karthik
>
> [1]
> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201611.mbox/%3CCALwhT97kAO6zuUjXTsmHgw_j-q-M9t9Gm9Vf11aogD%2B%2Bqgp2jQ%40mail.gmail.com%3E
> [2]
> https://github.com/apache/hadoop/commit/386f3a2af749bc96bdca1dc2302a3589773091dc
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org