[DISCUSS] Breaking upgrades as (not?) breaking SemVer promise

[Jarek Potiuk <ja...@potiuk.com>]

Hello everyone,

I wanted to separate one topic into a separate side discussion. Maybe
not useful with Dask discussion but might be something that we might
use for the future.

> This means that this change might potentially break existing
> setups for some users when they upgrade Airflow version, which contradicts
> the Airflow deprecation policy

also there was another commend from Daniel:

> As a user, your concerns and expectations regarding semver are about
> essentially how the code works, e.g. are you going to have to refactor all
> of your 500 dags. In other words the API.. But to me this is a lower
> evel thing that doesn't seem should strictly be forbidden. It's like...
> when we change the dependencies in airflow, inevitably that will break
> someone's cluster if they blindly deploy to prod. But this is not a
> prudent thing to do. So I think it's not an unreasonable expectation that a
> user verify that their cluster will spin up. It's something you can verify
> without running any dags at all (i.e. if you set dask executor and you
> start airflow, i think airflow will not start). And then users can rely on
> semver about changing behavior, which is not something you necessarily can
> tell without running dags.

I tend to agree with Daniel that technically speaking SemVer does not
tell anything about breaking upgrades. It does not really say "you
might continue upgrades the same way as before and things will work"
but  "when you upgrade <CORRECTLY> things will work the way they
worked". This is the promise of https://semver.org/ .

I re-read it again and could not find any promise that "your process
of upgrade is covered by SemVer promises". Interestingly I also found
out (I did not know) that our rule "breaking changes in our
dependencies is not breaking our compatibility" is explained there in
FAQ:

> That would be considered compatible since it does not affect
> the public API. Software that  explicitly depends on the same
> dependencies as your package should have their own
> dependency specifications and the author will
> notice any conflicts. Determining whether the change
> is a patch level or minor level modification depends on
> whether you updated your dependencies in order to fix a
> bug or introduce new functionality. We would usually expect
> additional code for the latter instance, in which case
> it’s obviously a minor level increment.


We do not make a promise that our upgrade process should look the same
between versions - it largely is the same, but IMHO technically we can
throw-in a new step or add a note in the upgrade process telling the
user to do things differently to upgrade.

IMHO it is perfectly fine if  in the release notes/significant section
we tell the user "If you used XX " you need to make sure you install
airflow with [xx] extra. Yes. It might break some installation scripts
and CI workflows, but upgrading to new version of Airflow should
ALWAYS be a deliberate action - with migration of the DB and reading
release notes before doing the upgrade - and failing such automated
workflow of upgrade is at most a signal for whoever maintains it to
make some fixes to their process.

I also think that guaranteed upgrades work is rather difficult - ys we
have our "reproducible" installation with constraints (which is
guaranteed to work), but there are many other ways and tools people
might install airflow - for example we have poetry or bazel
installation - and for example we already broke basel installation by
adding back limits on pre-installed providers, because basel expects
dependency graph to be DAG and for us (and pip/PyPI) having circles in
dependency graph are quite legitimate use case - see
https://github.com/bazelbuild/rules_python/pull/1166 - so technically
speaking we are not really ablet to guarantee much when it comes
"upgradeability".

But I would love to hear what others think about it. I would love it
if we get to a consensus on that and write it down in the "Public
Airflow Interface/SamVer" promise of ours so that we can communicate
it better rather than allow different people to have different
assumptions here.

I think we have two choices if we were to "disambiguate what Semver +
upgrades mean":

* either we agree that breaking upgrade flow is occasionally OK and
release notes should explain it

* or we document that the users can 100% rely on it working always
seamlessly when they upgrade - but I think this one is going to be
difficult to describe in the way that will include all the exclusions
such as the "basel" case.

J.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org

Re: [DISCUSS] Breaking upgrades as (not?) breaking SemVer promise

[Wei Lee <we...@gmail.com>]

I hope I'm not too late to join the discussion. From what I understand, I'd be OK to call this upgrade script change a non-breaking change as it doesn't change the public API. But again, different release managers might have other discretion here.



For the long term, I might vote for 

* Either we agree that breaking upgrade flow is occasionally OK and
release notes should explain it

based on how airflow versioning seems to work as of now.



If we choose the 100% strict option, we might need to allow/accept the MAJOR might be bumped more frequently. This might work for providers? But I'm not sure this is what's desired for the core airflow.


> On Jul 21, 2023, at 8:37 PM, Pankaj Koti <pa...@astronomer.io.INVALID> wrote:
> 
> The way I perceived it so far, semver and upgrades are tightly coupled and
> there
> might be users like me who perceive that way.
> 
> In the particular cases of moving a piece of code as an optional dependency,
> not keeping it part of the core or having it pre-installed, I agree we
> could have a
> breaking upgrade in a minor release as long as users don't have to change
> any
> DAG code but they only need to install additional dependencies and having
> well-written communication in the release notes would help reduce the
> change
> being interpreted as breaking.
> 
> In particular, if we agree with
> 
> 
> *`that breaking upgrade flow is occasionally OK andrelease notes should
> explain it*
> `
> would it make sense to reword slightly or remove the `strict` word here
> that
> mentions we follow strict semver here:
> https://github.com/apache/airflow/blob/49921763eb15f68f91da826a86690ba4c4155c35/README.md?plain=1#L243
> and give examples to reduce ambiguities where breaking upgrades are
> possible?
> 
> 
> 
> 
> Regards,
> 
> 
> 
> Pankaj Koti
> 
> *Senior Software Engineer, *OSS Engineering Team.
> Location: Pune, India
> 
> Timezone: Indian Standard Time (IST)
> 
> Email: pankaj.koti@astronomer.io
> 
> Mobile: +91 9730079985
> 
> 
> On Fri, Jul 21, 2023 at 2:48 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> 
>> Hello everyone,
>> 
>> I wanted to separate one topic into a separate side discussion. Maybe
>> not useful with Dask discussion but might be something that we might
>> use for the future.
>> 
>>> This means that this change might potentially break existing
>>> setups for some users when they upgrade Airflow version, which
>> contradicts
>>> the Airflow deprecation policy
>> 
>> also there was another commend from Daniel:
>> 
>>> As a user, your concerns and expectations regarding semver are about
>>> essentially how the code works, e.g. are you going to have to refactor
>> all
>>> of your 500 dags. In other words the API.. But to me this is a lower
>>> evel thing that doesn't seem should strictly be forbidden. It's like...
>>> when we change the dependencies in airflow, inevitably that will break
>>> someone's cluster if they blindly deploy to prod. But this is not a
>>> prudent thing to do. So I think it's not an unreasonable expectation
>> that a
>>> user verify that their cluster will spin up. It's something you can
>> verify
>>> without running any dags at all (i.e. if you set dask executor and you
>>> start airflow, i think airflow will not start). And then users can rely
>> on
>>> semver about changing behavior, which is not something you necessarily
>> can
>>> tell without running dags.
>> 
>> I tend to agree with Daniel that technically speaking SemVer does not
>> tell anything about breaking upgrades. It does not really say "you
>> might continue upgrades the same way as before and things will work"
>> but  "when you upgrade <CORRECTLY> things will work the way they
>> worked". This is the promise of https://semver.org/ .
>> 
>> I re-read it again and could not find any promise that "your process
>> of upgrade is covered by SemVer promises". Interestingly I also found
>> out (I did not know) that our rule "breaking changes in our
>> dependencies is not breaking our compatibility" is explained there in
>> FAQ:
>> 
>>> That would be considered compatible since it does not affect
>>> the public API. Software that  explicitly depends on the same
>>> dependencies as your package should have their own
>>> dependency specifications and the author will
>>> notice any conflicts. Determining whether the change
>>> is a patch level or minor level modification depends on
>>> whether you updated your dependencies in order to fix a
>>> bug or introduce new functionality. We would usually expect
>>> additional code for the latter instance, in which case
>>> it’s obviously a minor level increment.
>> 
>> 
>> We do not make a promise that our upgrade process should look the same
>> between versions - it largely is the same, but IMHO technically we can
>> throw-in a new step or add a note in the upgrade process telling the
>> user to do things differently to upgrade.
>> 
>> IMHO it is perfectly fine if  in the release notes/significant section
>> we tell the user "If you used XX " you need to make sure you install
>> airflow with [xx] extra. Yes. It might break some installation scripts
>> and CI workflows, but upgrading to new version of Airflow should
>> ALWAYS be a deliberate action - with migration of the DB and reading
>> release notes before doing the upgrade - and failing such automated
>> workflow of upgrade is at most a signal for whoever maintains it to
>> make some fixes to their process.
>> 
>> I also think that guaranteed upgrades work is rather difficult - ys we
>> have our "reproducible" installation with constraints (which is
>> guaranteed to work), but there are many other ways and tools people
>> might install airflow - for example we have poetry or bazel
>> installation - and for example we already broke basel installation by
>> adding back limits on pre-installed providers, because basel expects
>> dependency graph to be DAG and for us (and pip/PyPI) having circles in
>> dependency graph are quite legitimate use case - see
>> https://github.com/bazelbuild/rules_python/pull/1166 - so technically
>> speaking we are not really ablet to guarantee much when it comes
>> "upgradeability".
>> 
>> But I would love to hear what others think about it. I would love it
>> if we get to a consensus on that and write it down in the "Public
>> Airflow Interface/SamVer" promise of ours so that we can communicate
>> it better rather than allow different people to have different
>> assumptions here.
>> 
>> I think we have two choices if we were to "disambiguate what Semver +
>> upgrades mean":
>> 
>> * either we agree that breaking upgrade flow is occasionally OK and
>> release notes should explain it
>> 
>> * or we document that the users can 100% rely on it working always
>> seamlessly when they upgrade - but I think this one is going to be
>> difficult to describe in the way that will include all the exclusions
>> such as the "basel" case.
>> 
>> J.
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
>> For additional commands, e-mail: dev-help@airflow.apache.org
>> 
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org

Re: [DISCUSS] Breaking upgrades as (not?) breaking SemVer promise

[Pankaj Koti <pa...@astronomer.io.INVALID>]

The way I perceived it so far, semver and upgrades are tightly coupled and
there
might be users like me who perceive that way.

In the particular cases of moving a piece of code as an optional dependency,
not keeping it part of the core or having it pre-installed, I agree we
could have a
breaking upgrade in a minor release as long as users don't have to change
any
DAG code but they only need to install additional dependencies and having
well-written communication in the release notes would help reduce the
change
being interpreted as breaking.

In particular, if we agree with


*`that breaking upgrade flow is occasionally OK andrelease notes should
explain it*
`
would it make sense to reword slightly or remove the `strict` word here
that
mentions we follow strict semver here:
https://github.com/apache/airflow/blob/49921763eb15f68f91da826a86690ba4c4155c35/README.md?plain=1#L243
and give examples to reduce ambiguities where breaking upgrades are
possible?




Regards,



Pankaj Koti

*Senior Software Engineer, *OSS Engineering Team.
Location: Pune, India

Timezone: Indian Standard Time (IST)

Email: pankaj.koti@astronomer.io

Mobile: +91 9730079985


On Fri, Jul 21, 2023 at 2:48 PM Jarek Potiuk <ja...@potiuk.com> wrote:

> Hello everyone,
>
> I wanted to separate one topic into a separate side discussion. Maybe
> not useful with Dask discussion but might be something that we might
> use for the future.
>
> > This means that this change might potentially break existing
> > setups for some users when they upgrade Airflow version, which
> contradicts
> > the Airflow deprecation policy
>
> also there was another commend from Daniel:
>
> > As a user, your concerns and expectations regarding semver are about
> > essentially how the code works, e.g. are you going to have to refactor
> all
> > of your 500 dags. In other words the API.. But to me this is a lower
> > evel thing that doesn't seem should strictly be forbidden. It's like...
> > when we change the dependencies in airflow, inevitably that will break
> > someone's cluster if they blindly deploy to prod. But this is not a
> > prudent thing to do. So I think it's not an unreasonable expectation
> that a
> > user verify that their cluster will spin up. It's something you can
> verify
> > without running any dags at all (i.e. if you set dask executor and you
> > start airflow, i think airflow will not start). And then users can rely
> on
> > semver about changing behavior, which is not something you necessarily
> can
> > tell without running dags.
>
> I tend to agree with Daniel that technically speaking SemVer does not
> tell anything about breaking upgrades. It does not really say "you
> might continue upgrades the same way as before and things will work"
> but  "when you upgrade <CORRECTLY> things will work the way they
> worked". This is the promise of https://semver.org/ .
>
> I re-read it again and could not find any promise that "your process
> of upgrade is covered by SemVer promises". Interestingly I also found
> out (I did not know) that our rule "breaking changes in our
> dependencies is not breaking our compatibility" is explained there in
> FAQ:
>
> > That would be considered compatible since it does not affect
> > the public API. Software that  explicitly depends on the same
> > dependencies as your package should have their own
> > dependency specifications and the author will
> > notice any conflicts. Determining whether the change
> > is a patch level or minor level modification depends on
> > whether you updated your dependencies in order to fix a
> > bug or introduce new functionality. We would usually expect
> > additional code for the latter instance, in which case
> > it’s obviously a minor level increment.
>
>
> We do not make a promise that our upgrade process should look the same
> between versions - it largely is the same, but IMHO technically we can
> throw-in a new step or add a note in the upgrade process telling the
> user to do things differently to upgrade.
>
> IMHO it is perfectly fine if  in the release notes/significant section
> we tell the user "If you used XX " you need to make sure you install
> airflow with [xx] extra. Yes. It might break some installation scripts
> and CI workflows, but upgrading to new version of Airflow should
> ALWAYS be a deliberate action - with migration of the DB and reading
> release notes before doing the upgrade - and failing such automated
> workflow of upgrade is at most a signal for whoever maintains it to
> make some fixes to their process.
>
> I also think that guaranteed upgrades work is rather difficult - ys we
> have our "reproducible" installation with constraints (which is
> guaranteed to work), but there are many other ways and tools people
> might install airflow - for example we have poetry or bazel
> installation - and for example we already broke basel installation by
> adding back limits on pre-installed providers, because basel expects
> dependency graph to be DAG and for us (and pip/PyPI) having circles in
> dependency graph are quite legitimate use case - see
> https://github.com/bazelbuild/rules_python/pull/1166 - so technically
> speaking we are not really ablet to guarantee much when it comes
> "upgradeability".
>
> But I would love to hear what others think about it. I would love it
> if we get to a consensus on that and write it down in the "Public
> Airflow Interface/SamVer" promise of ours so that we can communicate
> it better rather than allow different people to have different
> assumptions here.
>
> I think we have two choices if we were to "disambiguate what Semver +
> upgrades mean":
>
> * either we agree that breaking upgrade flow is occasionally OK and
> release notes should explain it
>
> * or we document that the users can 100% rely on it working always
> seamlessly when they upgrade - but I think this one is going to be
> difficult to describe in the way that will include all the exclusions
> such as the "basel" case.
>
> J.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> For additional commands, e-mail: dev-help@airflow.apache.org
>
>