You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Jarek Potiuk <ja...@potiuk.com> on 2023/03/29 18:18:15 UTC
[LAZY CONSENSUS] Introduce suspension process for providers due to dependencies holding us back
Hello everyone,
Here is a formal ask for consensus on the new process of suspending
some providers that hold us back from upgrading old dependencies.
It has been discussed in
https://lists.apache.org/thread/j98bgw9jo7xr4fvjh27d6bfoyxr1omcm and
since it seems we have a consensus, I am calling for one (you do not
have to respond +1, this is what lazy consensus is about - but feel
free to do so).
The proposed wording is in the PR: https://github.com/apache/airflow/pull/30359
Copying it below for reference (might get slightly modified during review):
--------------------------
### Suspending releases for providers
In case a provider is found to require old dependencies that are not
compatible with upcoming versions of
the Apache Airflow or with newer dependencies required by other
providers, the provider's release
process can be suspended.
This means:
* The provider's status is set to "suspended"
* No new releases of the provider will be made until the problem with
dependencies is solved
* Sources of the provider remain in the repository for now (in the
future we might add process to remove them)
* No new changes will be accepted for the provider (other than the
ones that fix the dependencies)
* The provider will be removed from the list of Apache Airflow extras
in the next minor/major release
(2.7.0, 2.8.0, 3.0.0 etc.)
* Tests of the provider will not be run on our CI (in main branch)
* Dependencies of the provider will not be installed in our main
branch CI image nor included in constraints
The suspension might be triggered by any committer, providing that:
* The maintainers of dependencies of the provider are notified about
the issue and are given a reasonable
time to resolve it (at least 1 week)
* Other options to resolve the issue have been exhausted and there are
good reasons for upgrading
the old dependencies in question
* Explanation, why we need to suspend the provider is stated in a
public discussion in the devlist. Followed
by LAZY CONSENSUS or VOTE (with the majority of the committers
agreeing that we should suspend the provider)
The suspension will be lifted when the dependencies of the provider
are made compatible with the Apache
Airflow and with other providers.
J.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org
Re: [LAZY CONSENSUS] Introduce suspension process for providers due to dependencies holding us back
Posted by Jarek Potiuk <ja...@potiuk.com>.
I also implemented and tested a mechanism to suspend the providers: PR is
here: https://github.com/apache/airflow/pull/30422. Once this one and
https://github.com/apache/airflow/pull/30359 are merged, we should be able
to proceed with suspending providers that hold us back next week (for now
the first candidate is yandex, unless they fix the protobuf, following
https://github.com/yandex-cloud/python-sdk/issues/71 issue).
On Sat, Apr 1, 2023 at 4:39 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> Lazy consensus on suspension is reached.
>
> We already got a round of reviews on
> https://github.com/apache/airflow/pull/30359, once I get approval, I will
> merge. In the meantime I will prepare suspension mechanisms in our tooling.
>
> On Wed, Mar 29, 2023 at 8:37 PM Eugen Kosteev <eu...@kosteev.com> wrote:
>
>> +1
>>
>> On Wed, Mar 29, 2023 at 8:29 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>>
>> > The consensus will be reached in 72H on Saturday 1st of April 8.30pm
>> > CEST (unless there is a dissent).
>> >
>> > On Wed, Mar 29, 2023 at 8:18 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>> > >
>> > > Hello everyone,
>> > >
>> > > Here is a formal ask for consensus on the new process of suspending
>> > > some providers that hold us back from upgrading old dependencies.
>> > >
>> > > It has been discussed in
>> > > https://lists.apache.org/thread/j98bgw9jo7xr4fvjh27d6bfoyxr1omcm and
>> > > since it seems we have a consensus, I am calling for one (you do not
>> > > have to respond +1, this is what lazy consensus is about - but feel
>> > > free to do so).
>> > >
>> > > The proposed wording is in the PR:
>> > https://github.com/apache/airflow/pull/30359
>> > >
>> > > Copying it below for reference (might get slightly modified during
>> > review):
>> > >
>> > > --------------------------
>> > >
>> > > ### Suspending releases for providers
>> > >
>> > > In case a provider is found to require old dependencies that are not
>> > > compatible with upcoming versions of
>> > > the Apache Airflow or with newer dependencies required by other
>> > > providers, the provider's release
>> > > process can be suspended.
>> > >
>> > > This means:
>> > >
>> > > * The provider's status is set to "suspended"
>> > > * No new releases of the provider will be made until the problem with
>> > > dependencies is solved
>> > > * Sources of the provider remain in the repository for now (in the
>> > > future we might add process to remove them)
>> > > * No new changes will be accepted for the provider (other than the
>> > > ones that fix the dependencies)
>> > > * The provider will be removed from the list of Apache Airflow extras
>> > > in the next minor/major release
>> > > (2.7.0, 2.8.0, 3.0.0 etc.)
>> > > * Tests of the provider will not be run on our CI (in main branch)
>> > > * Dependencies of the provider will not be installed in our main
>> > > branch CI image nor included in constraints
>> > >
>> > > The suspension might be triggered by any committer, providing that:
>> > >
>> > > * The maintainers of dependencies of the provider are notified about
>> > > the issue and are given a reasonable
>> > > time to resolve it (at least 1 week)
>> > > * Other options to resolve the issue have been exhausted and there are
>> > > good reasons for upgrading
>> > > the old dependencies in question
>> > > * Explanation, why we need to suspend the provider is stated in a
>> > > public discussion in the devlist. Followed
>> > > by LAZY CONSENSUS or VOTE (with the majority of the committers
>> > > agreeing that we should suspend the provider)
>> > >
>> > > The suspension will be lifted when the dependencies of the provider
>> > > are made compatible with the Apache
>> > > Airflow and with other providers.
>> > >
>> > > J.
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
>> > For additional commands, e-mail: dev-help@airflow.apache.org
>> >
>> >
>>
>> --
>> Eugene
>>
>
Re: [LAZY CONSENSUS] Introduce suspension process for providers due to dependencies holding us back
Posted by Jarek Potiuk <ja...@potiuk.com>.
Lazy consensus on suspension is reached.
We already got a round of reviews on
https://github.com/apache/airflow/pull/30359, once I get approval, I will
merge. In the meantime I will prepare suspension mechanisms in our tooling.
On Wed, Mar 29, 2023 at 8:37 PM Eugen Kosteev <eu...@kosteev.com> wrote:
> +1
>
> On Wed, Mar 29, 2023 at 8:29 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
> > The consensus will be reached in 72H on Saturday 1st of April 8.30pm
> > CEST (unless there is a dissent).
> >
> > On Wed, Mar 29, 2023 at 8:18 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> > >
> > > Hello everyone,
> > >
> > > Here is a formal ask for consensus on the new process of suspending
> > > some providers that hold us back from upgrading old dependencies.
> > >
> > > It has been discussed in
> > > https://lists.apache.org/thread/j98bgw9jo7xr4fvjh27d6bfoyxr1omcm and
> > > since it seems we have a consensus, I am calling for one (you do not
> > > have to respond +1, this is what lazy consensus is about - but feel
> > > free to do so).
> > >
> > > The proposed wording is in the PR:
> > https://github.com/apache/airflow/pull/30359
> > >
> > > Copying it below for reference (might get slightly modified during
> > review):
> > >
> > > --------------------------
> > >
> > > ### Suspending releases for providers
> > >
> > > In case a provider is found to require old dependencies that are not
> > > compatible with upcoming versions of
> > > the Apache Airflow or with newer dependencies required by other
> > > providers, the provider's release
> > > process can be suspended.
> > >
> > > This means:
> > >
> > > * The provider's status is set to "suspended"
> > > * No new releases of the provider will be made until the problem with
> > > dependencies is solved
> > > * Sources of the provider remain in the repository for now (in the
> > > future we might add process to remove them)
> > > * No new changes will be accepted for the provider (other than the
> > > ones that fix the dependencies)
> > > * The provider will be removed from the list of Apache Airflow extras
> > > in the next minor/major release
> > > (2.7.0, 2.8.0, 3.0.0 etc.)
> > > * Tests of the provider will not be run on our CI (in main branch)
> > > * Dependencies of the provider will not be installed in our main
> > > branch CI image nor included in constraints
> > >
> > > The suspension might be triggered by any committer, providing that:
> > >
> > > * The maintainers of dependencies of the provider are notified about
> > > the issue and are given a reasonable
> > > time to resolve it (at least 1 week)
> > > * Other options to resolve the issue have been exhausted and there are
> > > good reasons for upgrading
> > > the old dependencies in question
> > > * Explanation, why we need to suspend the provider is stated in a
> > > public discussion in the devlist. Followed
> > > by LAZY CONSENSUS or VOTE (with the majority of the committers
> > > agreeing that we should suspend the provider)
> > >
> > > The suspension will be lifted when the dependencies of the provider
> > > are made compatible with the Apache
> > > Airflow and with other providers.
> > >
> > > J.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > For additional commands, e-mail: dev-help@airflow.apache.org
> >
> >
>
> --
> Eugene
>
Re: [LAZY CONSENSUS] Introduce suspension process for providers due to dependencies holding us back
Posted by Eugen Kosteev <eu...@kosteev.com>.
+1
On Wed, Mar 29, 2023 at 8:29 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> The consensus will be reached in 72H on Saturday 1st of April 8.30pm
> CEST (unless there is a dissent).
>
> On Wed, Mar 29, 2023 at 8:18 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> >
> > Hello everyone,
> >
> > Here is a formal ask for consensus on the new process of suspending
> > some providers that hold us back from upgrading old dependencies.
> >
> > It has been discussed in
> > https://lists.apache.org/thread/j98bgw9jo7xr4fvjh27d6bfoyxr1omcm and
> > since it seems we have a consensus, I am calling for one (you do not
> > have to respond +1, this is what lazy consensus is about - but feel
> > free to do so).
> >
> > The proposed wording is in the PR:
> https://github.com/apache/airflow/pull/30359
> >
> > Copying it below for reference (might get slightly modified during
> review):
> >
> > --------------------------
> >
> > ### Suspending releases for providers
> >
> > In case a provider is found to require old dependencies that are not
> > compatible with upcoming versions of
> > the Apache Airflow or with newer dependencies required by other
> > providers, the provider's release
> > process can be suspended.
> >
> > This means:
> >
> > * The provider's status is set to "suspended"
> > * No new releases of the provider will be made until the problem with
> > dependencies is solved
> > * Sources of the provider remain in the repository for now (in the
> > future we might add process to remove them)
> > * No new changes will be accepted for the provider (other than the
> > ones that fix the dependencies)
> > * The provider will be removed from the list of Apache Airflow extras
> > in the next minor/major release
> > (2.7.0, 2.8.0, 3.0.0 etc.)
> > * Tests of the provider will not be run on our CI (in main branch)
> > * Dependencies of the provider will not be installed in our main
> > branch CI image nor included in constraints
> >
> > The suspension might be triggered by any committer, providing that:
> >
> > * The maintainers of dependencies of the provider are notified about
> > the issue and are given a reasonable
> > time to resolve it (at least 1 week)
> > * Other options to resolve the issue have been exhausted and there are
> > good reasons for upgrading
> > the old dependencies in question
> > * Explanation, why we need to suspend the provider is stated in a
> > public discussion in the devlist. Followed
> > by LAZY CONSENSUS or VOTE (with the majority of the committers
> > agreeing that we should suspend the provider)
> >
> > The suspension will be lifted when the dependencies of the provider
> > are made compatible with the Apache
> > Airflow and with other providers.
> >
> > J.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> For additional commands, e-mail: dev-help@airflow.apache.org
>
>
--
Eugene
Re: [LAZY CONSENSUS] Introduce suspension process for providers due to dependencies holding us back
Posted by Jarek Potiuk <ja...@potiuk.com>.
The consensus will be reached in 72H on Saturday 1st of April 8.30pm
CEST (unless there is a dissent).
On Wed, Mar 29, 2023 at 8:18 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
> Hello everyone,
>
> Here is a formal ask for consensus on the new process of suspending
> some providers that hold us back from upgrading old dependencies.
>
> It has been discussed in
> https://lists.apache.org/thread/j98bgw9jo7xr4fvjh27d6bfoyxr1omcm and
> since it seems we have a consensus, I am calling for one (you do not
> have to respond +1, this is what lazy consensus is about - but feel
> free to do so).
>
> The proposed wording is in the PR: https://github.com/apache/airflow/pull/30359
>
> Copying it below for reference (might get slightly modified during review):
>
> --------------------------
>
> ### Suspending releases for providers
>
> In case a provider is found to require old dependencies that are not
> compatible with upcoming versions of
> the Apache Airflow or with newer dependencies required by other
> providers, the provider's release
> process can be suspended.
>
> This means:
>
> * The provider's status is set to "suspended"
> * No new releases of the provider will be made until the problem with
> dependencies is solved
> * Sources of the provider remain in the repository for now (in the
> future we might add process to remove them)
> * No new changes will be accepted for the provider (other than the
> ones that fix the dependencies)
> * The provider will be removed from the list of Apache Airflow extras
> in the next minor/major release
> (2.7.0, 2.8.0, 3.0.0 etc.)
> * Tests of the provider will not be run on our CI (in main branch)
> * Dependencies of the provider will not be installed in our main
> branch CI image nor included in constraints
>
> The suspension might be triggered by any committer, providing that:
>
> * The maintainers of dependencies of the provider are notified about
> the issue and are given a reasonable
> time to resolve it (at least 1 week)
> * Other options to resolve the issue have been exhausted and there are
> good reasons for upgrading
> the old dependencies in question
> * Explanation, why we need to suspend the provider is stated in a
> public discussion in the devlist. Followed
> by LAZY CONSENSUS or VOTE (with the majority of the committers
> agreeing that we should suspend the provider)
>
> The suspension will be lifted when the dependencies of the provider
> are made compatible with the Apache
> Airflow and with other providers.
>
> J.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org