Upcoming provider's release and Airflow 2.1+ compatibility

[Jarek Potiuk <ja...@potiuk.com>]

Hello everyone,

I am working on a June provider's release - I plan to release RC latest
early next week but there is one thing that is different in this release
that I wanted to make everyone aware of.

Since we got rid of "@apply_defaults" decorators needed in 2.1, we also got
rid of it in all providers and it makes those providers 2.1+ compatible
only.

With the current approach of releasing providers this means that:

* new providers will be 2.1+ compatible only (apache-airflow>=2.1.0 in
install_requires for those providers).

* we need to release ALL providers with a new MAJOR version to indicate
breaking changes (so that people will not accidentally pull-in Airflow 2.1
by just upgrading patchlevel or minor version of a provider). Those won't
be breaking changes for the providers/operators but rather the need to use
Airflow 2.1 makes it "breaking change".

* It also opens-up the providers for some new features - for example in 2.1
we've added DBAPIHook's "result handler" which we now will be able to use
in DB Providers to use per-db functionality (in Snowflake and Oracle
provider for example).

The existing providers will continue to work also for 2.1, but when you
want to use the latest released providers, you will have to upgrade to
Airlfow 2.1. This is actually a good thing, because the provider usage
might drive faster adoption of the 2.1 version of Airflow. The 2.1 is by
definition (SEMVER) backwards-compatible with 2.0 so there should be no big
problems with migration.

This has been discussed as one of the scenarios when we agreed to adopt
separate-providers approach during 2.0 release, but since this is the first
time it happens, I thought it's good to explicitly state it in the devlist.

Unless there are some strong objections from the community and good reasons
to approach it differently, I propose to adopt (and maybe even write down
some explicit rules about it).

J.

-- 
+48 660 796 129

Re: Upcoming provider's release and Airflow 2.1+ compatibility

["...." <dz...@gmail.com>]

Yeah.  it drives faster Airflow evolutions while maintaining backward
compatibility.Who needs it, can keep the environment more stable, but who
cares about new features can get them faster.



On Wed, May 26, 2021, 12:06 Jarek Potiuk <ja...@potiuk.com> wrote:

> Hello everyone,
>
> I am working on a June provider's release - I plan to release RC latest
> early next week but there is one thing that is different in this release
> that I wanted to make everyone aware of.
>
> Since we got rid of "@apply_defaults" decorators needed in 2.1, we also
> got rid of it in all providers and it makes those providers 2.1+ compatible
> only.
>
> With the current approach of releasing providers this means that:
>
> * new providers will be 2.1+ compatible only (apache-airflow>=2.1.0 in
> install_requires for those providers).
>
> * we need to release ALL providers with a new MAJOR version to indicate
> breaking changes (so that people will not accidentally pull-in Airflow 2.1
> by just upgrading patchlevel or minor version of a provider). Those won't
> be breaking changes for the providers/operators but rather the need to use
> Airflow 2.1 makes it "breaking change".
>
> * It also opens-up the providers for some new features - for example in
> 2.1 we've added DBAPIHook's "result handler" which we now will be able to
> use in DB Providers to use per-db functionality (in Snowflake and Oracle
> provider for example).
>
> The existing providers will continue to work also for 2.1, but when you
> want to use the latest released providers, you will have to upgrade to
> Airlfow 2.1. This is actually a good thing, because the provider usage
> might drive faster adoption of the 2.1 version of Airflow. The 2.1 is by
> definition (SEMVER) backwards-compatible with 2.0 so there should be no big
> problems with migration.
>
> This has been discussed as one of the scenarios when we agreed to adopt
> separate-providers approach during 2.0 release, but since this is the first
> time it happens, I thought it's good to explicitly state it in the devlist.
>
> Unless there are some strong objections from the community and good
> reasons to approach it differently, I propose to adopt (and maybe even
> write down some explicit rules about it).
>
> J.
>
> --
> +48 660 796 129
>