Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

[Jarek Potiuk <ja...@potiuk.com>]

> maybe we should instead try to pull back just a bit on what we promise concerning backcompat.

Yes. Full agreement.


On Fri, Jan 27, 2023 at 8:20 PM Daniel Standish
<da...@astronomer.io.invalid> wrote:
>
> Thinking about this some more.
>
> As an airflow developer, a lot of our backcompat concerns (which takes up a substantial portion of our energy), is about the concern that we might break something for someone who has extended either built-in classes or provider classes.  Maybe we are to permissive in this way.  Following the example with executors, maybe we should instead try to pull back just a bit on what we promise concerning backcompat.
>
> Take for example secrets backend.  Perhaps what should be public is BaseSecretsBackend -- the interface -- and not any of the implementations we offer such as the AWS SecretsManagerBackend.  And again, BaseSecretsBackend is the interface, and its methods, when public, are part of the public interface, and subject to semver.  However, instance, such as secrets manager, are subject to semver with regard to their user-facing behavior, but not with regard to their internal structure, which is implementation detail.  In other words, AWS SecretsManagerBackend is provided as an end product, not as a library to be inhereted, and is not itself another public interface.  And of course users are free to extend this code, but we simply make no promises with recard to backcompat for these implementations.
>
> And the same could be true of most of our provider operators -- they are end products, and their user-facing behavior is subject to semver, but the specific code and implementation details are not subject to semver and its assumed that if you extend them then you do so with the knowledge that you have to either vendor in the operator code or just test before ugprades and make changes accordingly etc.
>
> WDYT?