You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Ash Berlin-Taylor <as...@apache.org> on 2020/03/17 21:40:53 UTC
[VOTE] AIP-15: Support Multiple-Schedulers for HA & Better
Scheduling Performance
Hi all,
This email calls for a vote on the design proposed in AIP-15, found here
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103092651
A few notes
- The proposed architecture is to use an active/active architecture
where each scheduler is fully capable
- Nothing in this proposal locks us in to this design long term, and if
we find it doesn't work or scale we can change it.
- Lock contention doesn't worry me, as we (plan to) skip locked rows,
meaning that when one scheduler has a DAG locked it will be skipped
over by other schedulers
This vote will last for 72 hours until 2020-03-20T21:40Z, and until at
least 3 votes have been cast.
This is my +1 vote.
Thanks,
Ash
[RESULT] [VOTE] AIP-15: Support Multiple-Schedulers for HA &
Better Scheduling Performance
Posted by Ash Berlin-Taylor <as...@apache.org>.
This vote has passed with 5 +1 votes and no -1 votes.
Work will begin on this soon :)
-ash
On Mar 19 2020, at 8:53 pm, Maxime Beauchemin
<ma...@gmail.com> wrote:
> +1 (binding)
>
> Solid work!
>
> On Tue, Mar 17, 2020 at 10:43 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>> +1 (binding)
>>
>> On Tue, Mar 17, 2020 at 11:16 PM Kaxil Naik <ka...@gmail.com> wrote:
>> >
>> > +1 (binding)
>> >
>> > On Tue, Mar 17, 2020 at 10:06 PM Deng Xiaodong <xd...@gmail.com>
>> wrote:
>> >
>> > > +1 (binding).
>> > >
>> > > Thanks for proceeding this AIP, Ash.
>> > >
>> > >
>> > > XD
>> > >
>> > > On Tue, Mar 17, 2020 at 22:40 Ash Berlin-Taylor <as...@apache.org>
>> wrote:
>> > >
>> > > > Hi all,
>> > > >
>> > > > This email calls for a vote on the design proposed in AIP-15, found
>> here
>> > > >
>> > > >
>> > > >
>> > >
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103092651
>> > > >
>> > > > A few notes
>> > > >
>> > > > - The proposed architecture is to use an active/active architecture
>> > > > where each scheduler is fully capable
>> > > > - Nothing in this proposal locks us in to this design long
>> term, and
>> if
>> > > > we find it doesn't work or scale we can change it.
>> > > > - Lock contention doesn't worry me, as we (plan to) skip locked rows,
>> > > > meaning that when one scheduler has a DAG locked it will be skipped
>> > > > over by other schedulers
>> > > >
>> > > > This vote will last for 72 hours until 2020-03-20T21:40Z, and until
>> at
>> > > > least 3 votes have been cast.
>> > > >
>> > > > This is my +1 vote.
>> > > >
>> > > > Thanks,
>> > > > Ash
>> > > >
>> > > >
>> > >
>>
>>
>>
>> --
>>
>> Jarek Potiuk
>> Polidea | Principal Software Engineer
>>
>> M: +48 660 796 129
>>
>
Re: [VOTE] AIP-15: Support Multiple-Schedulers for HA & Better
Scheduling Performance
Posted by Maxime Beauchemin <ma...@gmail.com>.
+1 (binding)
Solid work!
On Tue, Mar 17, 2020 at 10:43 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> +1 (binding)
>
> On Tue, Mar 17, 2020 at 11:16 PM Kaxil Naik <ka...@gmail.com> wrote:
> >
> > +1 (binding)
> >
> > On Tue, Mar 17, 2020 at 10:06 PM Deng Xiaodong <xd...@gmail.com>
> wrote:
> >
> > > +1 (binding).
> > >
> > > Thanks for proceeding this AIP, Ash.
> > >
> > >
> > > XD
> > >
> > > On Tue, Mar 17, 2020 at 22:40 Ash Berlin-Taylor <as...@apache.org>
> wrote:
> > >
> > > > Hi all,
> > > >
> > > > This email calls for a vote on the design proposed in AIP-15, found
> here
> > > >
> > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103092651
> > > >
> > > > A few notes
> > > >
> > > > - The proposed architecture is to use an active/active architecture
> > > > where each scheduler is fully capable
> > > > - Nothing in this proposal locks us in to this design long term, and
> if
> > > > we find it doesn't work or scale we can change it.
> > > > - Lock contention doesn't worry me, as we (plan to) skip locked rows,
> > > > meaning that when one scheduler has a DAG locked it will be skipped
> > > > over by other schedulers
> > > >
> > > > This vote will last for 72 hours until 2020-03-20T21:40Z, and until
> at
> > > > least 3 votes have been cast.
> > > >
> > > > This is my +1 vote.
> > > >
> > > > Thanks,
> > > > Ash
> > > >
> > > >
> > >
>
>
>
> --
>
> Jarek Potiuk
> Polidea | Principal Software Engineer
>
> M: +48 660 796 129
>
Re: [VOTE] AIP-15: Support Multiple-Schedulers for HA & Better
Scheduling Performance
Posted by Jarek Potiuk <Ja...@polidea.com>.
+1 (binding)
On Tue, Mar 17, 2020 at 11:16 PM Kaxil Naik <ka...@gmail.com> wrote:
>
> +1 (binding)
>
> On Tue, Mar 17, 2020 at 10:06 PM Deng Xiaodong <xd...@gmail.com> wrote:
>
> > +1 (binding).
> >
> > Thanks for proceeding this AIP, Ash.
> >
> >
> > XD
> >
> > On Tue, Mar 17, 2020 at 22:40 Ash Berlin-Taylor <as...@apache.org> wrote:
> >
> > > Hi all,
> > >
> > > This email calls for a vote on the design proposed in AIP-15, found here
> > >
> > >
> > >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103092651
> > >
> > > A few notes
> > >
> > > - The proposed architecture is to use an active/active architecture
> > > where each scheduler is fully capable
> > > - Nothing in this proposal locks us in to this design long term, and if
> > > we find it doesn't work or scale we can change it.
> > > - Lock contention doesn't worry me, as we (plan to) skip locked rows,
> > > meaning that when one scheduler has a DAG locked it will be skipped
> > > over by other schedulers
> > >
> > > This vote will last for 72 hours until 2020-03-20T21:40Z, and until at
> > > least 3 votes have been cast.
> > >
> > > This is my +1 vote.
> > >
> > > Thanks,
> > > Ash
> > >
> > >
> >
--
Jarek Potiuk
Polidea | Principal Software Engineer
M: +48 660 796 129
Re: [VOTE] AIP-15: Support Multiple-Schedulers for HA & Better
Scheduling Performance
Posted by Kaxil Naik <ka...@gmail.com>.
+1 (binding)
On Tue, Mar 17, 2020 at 10:06 PM Deng Xiaodong <xd...@gmail.com> wrote:
> +1 (binding).
>
> Thanks for proceeding this AIP, Ash.
>
>
> XD
>
> On Tue, Mar 17, 2020 at 22:40 Ash Berlin-Taylor <as...@apache.org> wrote:
>
> > Hi all,
> >
> > This email calls for a vote on the design proposed in AIP-15, found here
> >
> >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103092651
> >
> > A few notes
> >
> > - The proposed architecture is to use an active/active architecture
> > where each scheduler is fully capable
> > - Nothing in this proposal locks us in to this design long term, and if
> > we find it doesn't work or scale we can change it.
> > - Lock contention doesn't worry me, as we (plan to) skip locked rows,
> > meaning that when one scheduler has a DAG locked it will be skipped
> > over by other schedulers
> >
> > This vote will last for 72 hours until 2020-03-20T21:40Z, and until at
> > least 3 votes have been cast.
> >
> > This is my +1 vote.
> >
> > Thanks,
> > Ash
> >
> >
>
Re: [VOTE] AIP-15: Support Multiple-Schedulers for HA & Better
Scheduling Performance
Posted by Deng Xiaodong <xd...@gmail.com>.
+1 (binding).
Thanks for proceeding this AIP, Ash.
XD
On Tue, Mar 17, 2020 at 22:40 Ash Berlin-Taylor <as...@apache.org> wrote:
> Hi all,
>
> This email calls for a vote on the design proposed in AIP-15, found here
>
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103092651
>
> A few notes
>
> - The proposed architecture is to use an active/active architecture
> where each scheduler is fully capable
> - Nothing in this proposal locks us in to this design long term, and if
> we find it doesn't work or scale we can change it.
> - Lock contention doesn't worry me, as we (plan to) skip locked rows,
> meaning that when one scheduler has a DAG locked it will be skipped
> over by other schedulers
>
> This vote will last for 72 hours until 2020-03-20T21:40Z, and until at
> least 3 votes have been cast.
>
> This is my +1 vote.
>
> Thanks,
> Ash
>
>