[LAZY CONSENSUS] Do not treat min-airflow-version bump in providers as breaking

[Jarek Potiuk <ja...@potiuk.com>]

Hey everyone,

Following the explanation in
https://lists.apache.org/thread/xpngxsdxmk1vw2wk34py8sdsqfmjdw9g  I would
like to call for a lazy consensus on the change proposed in the PR
https://github.com/apache/airflow/pull/27613 for the November wave of
providers.

When we bumped "min airflow version" to 2.2+ we bumped all provider's MAJOR
versions signalling a breaking change. But I realized since this is not
really needed - because the change is not really breaking the user's
workflows (explanation in the linked discussion)

I would like to call for a lazy consensus on that - in parallel to
releasing the providers - we can always cancel RCs if we release them so
hopefully the consensus will be reached before we actually release the
providers.

Of course, if there are objections and some reasons why this is not a good
approach, we can always revisit this, cancel the release candidates and
re-do the release with all MAJOR versions.

J.

Re: [LAZY CONSENSUS] Do not treat min-airflow-version bump in providers as breaking

[Jarek Potiuk <ja...@potiuk.com>]

No opposition. Lazy Consensus agreed. I release providers accordingly.

On Mon, Nov 14, 2022 at 10:31 PM Daniel Standish
<da...@astronomer.io.invalid> wrote:
>
> I think it makes sense and I'm a +1.
>
> For the convenience of other readers I'll paste your rationale here:
>
>> The rationale i have - that from the point of view of provider, it's just a dependency change (which we generally consided non-breaking) and it does not break people's workflows in general at all.
>> If you are on Airflow 2.2, you cannot install this release anyway. So the change is not breking - from the point of view of Airflow 2.2 user, there is no change. The user will have to continue using old provider.
>> If you are on Airflow 2.3, you install it and it "just works". No breaking in your workflow whatsoever.
>> There are two questions/potential problems/considerations here:
>> You could accidentally bring your airflow up if you have Airflow 2.2 and force installation of 2.3+provider. I'd argue it's a problem of the user and the most we can do it is to warn user about it in the documentation. This is also true for many of our dependencies. There are a number of dependencies that - when upgraded independently - might bring airlfow version up (for example sqlalchemy bump to 1.4 will upgrade airflow to 2.3). Unfortunately pip works like that and we cannot do anything about it - everything that happens there happens long before any of our code is executed and we cannot even give a warning to the user when the user attempts to do so. Here potentially the "major" bump might give the user a pause, but I argue that this is a very weak signal - the major bump should signal incompatibilities in the provider itself, not potential problems with Airflow. Luckily this is not destructive nowadays. User will get information about migrations not applied and will be able to recover.
>> I was thinking whether it shoudl be a "feature" or "patchlevel" bump. I chose "feature" pretty arbitrary here and without any good "reason". SemVer does not really address this case so we are free to choose whatever we feel is appropriate. And I think personally that signalling minor bump is a bit more prominent. We do not (I think) make it a breaking change (see above), but we also do not want to make it a "meh" change. We need to pick some scheme that will allow us to publish it as a new package in PyPI even without any changes. So I chose "feature". But it could be "patchlevel" as well. Both "patchlevel" and "feature" are not breaking (probably) user's workflow so choosing one over another is pretty arbitrary decision.
>
>

Re: [LAZY CONSENSUS] Do not treat min-airflow-version bump in providers as breaking

[Daniel Standish <da...@astronomer.io.INVALID>]

I think it makes sense and I'm a +1.

For the convenience of other readers I'll paste your rationale here:

The rationale i have - that from the point of view of provider, it's just a
> dependency change (which we generally consided non-breaking) and it does
> not break people's workflows in general at all.
> If you are on Airflow 2.2, you cannot install this release anyway. So the
> change is not breking - from the point of view of Airflow 2.2 user, there
> is no change. The user will have to continue using old provider.
> If you are on Airflow 2.3, you install it and it "just works". No breaking
> in your workflow whatsoever.
> There are two questions/potential problems/considerations here:
> You could accidentally bring your airflow up if you have Airflow 2.2 and
> force installation of 2.3+provider. I'd argue it's a problem of the user
> and the most we can do it is to warn user about it in the documentation.
> This is also true for many of our dependencies. There are a number of
> dependencies that - when upgraded independently - might bring airlfow
> version up (for example sqlalchemy bump to 1.4 will upgrade airflow to
> 2.3). Unfortunately pip works like that and we cannot do anything about
> it - everything that happens there happens long before any of our code is
> executed and we cannot even give a warning to the user when the user
> attempts to do so. Here potentially the "major" bump might give the user a
> pause, but I argue that this is a very weak signal - the major bump should
> signal incompatibilities in the provider itself, not potential problems
> with Airflow. Luckily this is not destructive nowadays. User will get
> information about migrations not applied and will be able to recover.
> I was thinking whether it shoudl be a "feature" or "patchlevel" bump. I
> chose "feature" pretty arbitrary here and without any good "reason". SemVer
> does not really address this case so we are free to choose whatever we feel
> is appropriate. And I think personally that signalling minor bump is a bit
> more prominent. We do not (I think) make it a breaking change (see above),
> but we also do not want to make it a "meh" change. We need to pick some
> scheme that will allow us to publish it as a new package in PyPI even
> without any changes. So I chose "feature". But it could be "patchlevel" as
> well. Both "patchlevel" and "feature" are not breaking (probably) user's
> workflow so choosing one over another is pretty arbitrary decision.

Re: [LAZY CONSENSUS] Do not treat min-airflow-version bump in providers as breaking

[Jarek Potiuk <ja...@potiuk.com>]

I hope to have the RC release for the November wave of providers
todays (pending https://github.com/apache/airflow/pull/27613 approval)
and unless someone has a problem with treating the min-version bump
approach as problematic, I would proceed assuming tha the lazy
consensus passed.

On Fri, Nov 11, 2022 at 2:28 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
> Hey everyone,
>
> Following the explanation in https://lists.apache.org/thread/xpngxsdxmk1vw2wk34py8sdsqfmjdw9g  I would like to call for a lazy consensus on the change proposed in the PR https://github.com/apache/airflow/pull/27613 for the November wave of providers.
>
> When we bumped "min airflow version" to 2.2+ we bumped all provider's MAJOR versions signalling a breaking change. But I realized since this is not really needed - because the change is not really breaking the user's workflows (explanation in the linked discussion)
>
> I would like to call for a lazy consensus on that - in parallel to releasing the providers - we can always cancel RCs if we release them so hopefully the consensus will be reached before we actually release the providers.
>
> Of course, if there are objections and some reasons why this is not a good approach, we can always revisit this, cancel the release candidates and re-do the release with all MAJOR versions.
>
> J.
>