[PROPOSAL] Provider's mixed governance model - first step of provider separation

[Jarek Potiuk <ja...@potiuk.com>]

Hello everyone,

This is a follow-up after a few discussions started about providers that
were put on hold around  the summit. I held a number of discussions during
theSummit and after, and as result I think I have a proposal that can move
forward some of the "stalled" decisions we need to make.

*TL;DR;*

My proposal is a "mixed-governance" model where "stakeholders" are more
responsible for cherry-picking and testing their providers (including
system testing) while Airflow PMC members will continue to be responsible
for releasing them.

*Why do we need that?*

Google, Amazon and possibly others teams who are interested in maintaining
more backwards compatible versions of their providers will commit to make
PRs of the cherry-picks for older release branches of their providers.
Those providers we release in parallel with the latest versions during the
normal provider cycle. We can deprecate changes more aggressively in the
"latest" release if we do that.

Those cherry-picked PRs will be driven, tested and performed by the
stakeholder teams (Google/Amazon, Databricks, others) and will only contain
cherry-picks, while we - as PMC - will release them following the ASF rules
(this is very important for the ASF to follow strict release policies
regarding who and how performs releases).

This also allows us to introduce similar rules for new provider's
acceptance for new providers for "main releases". It also allows running
the "system tests" for the provider under control of the stakeholder (after
applying AIP-47 changes).

*Example 1*: Google team can cherry-pick changes to a google-provider-6
branch and then we release a google 6.8.1 or 6.9.0 provider with some of
the bug-fixes and features (together with - say latest 8.1.0).

*Example 2*: DataLake provider from Databricks - can get accepted if
Databricks commits to maintaining it. We will release the provider as long
as Databricks maintains it.

*Longer context:*

I have - in my mind so far - a longer roadmap for providers that will lead
them to be separated from the core and I want to write an AIP about that
soon. This AIP will detail all the steps needed - I will work with multiple
interested parties on it and it will take some time to agree and complete.
But I want to start with something tangible that will solve quite a few
problems that were raised recently and something that seems to be possible
to be solved in the current provider release cycle (till the end of June)
and test some of the governance approach.

This proposal simply builds on our semver approach - we do not change it,
we just start releasing some providers (those that have some backing from
stakeholders) in more than one version - including "latest" and earlier,
more backwards-compatible branches. Not all providers - just some. Not all
branches - just those that the stakeholders will commit to maintain.

We need such a commitment from stakeholders, because we - as the
Airflow community and maintainers, want to only actively maintain the
latest releases, where it is in the interest of the stakeholders to
cherry-pick and test also earlier, more backwards compatible releases of
their choice.

*What problems this proposal solves:*

** Problem 1*: DEPRECATION REMOVAL

we can remove deprecations faster in "main" versions of the providers - no
need to introduce a deprecation policy - the stakeholders for the providers
will take care about cherry-picking and maintaining more
"backwards-compatible" versions. We are free to remove deprecations in
major releases (in a cherry-pickable way of course).

** Problem 2*: PROVIDERS DIVERGENCE

We avoid the problem (already happened with the composer release) that the
stakeholders in a given provider had to release their own version which was
not available in their community - with some cherry-picks. We want to avoid
"diverging" there - by releasing the cherry-picked providers by the
community, we also give other users an opportunity to follow "slower"
deprecation policies for as long as it is maintained.

** Problem 3*: PROVIDERS GOVERNANCE MODEL

We are going to test a governance model that we might apply when we split
providers. We are talking about it for quite some time - but this is what
helps us to test the model where stakeholders provide more "maintenance"
while the community still takes care about releases. We (as community) can
commit to releasing such a version of a provider as long as the stakeholder
will actively maintain it. We can stop at any moment if we do not have
support from the stakeholder. If it works - we can keep it as a long-term
solution. In the future we can think of other scenarios (passing ownership
of a provider to stakeholders who want it - providing we want it too) but
we can decide about it when we learn from the mixed-governance model and
see if it works.

** Problem 4*: ACCEPTING NEW PROVIDERS

If this is an acceptable approach - we can also apply a very
similar governance model to adding new providers and that should unblock
some of the PRs that are waiting for our decision. Knowing that we are
going to split and that we can expect "commitment" from a stakeholder, we
should be able to accept new providers. This might be possible assuming
that the stakeholder will make a similar commitment - but for new
providers, that commitment might also have to cover reviewing and testing
new changes. We might also decide as a community to stop releasing new
providers there if such support is missing. This way we can set the
expectations we have as a community for new providers - we will release
them as long as the stakeholder will actively make sure it is maintained.

** Problem 5.* SPLITTING PROVIDERS FROM CORE

We all know we want to split providers from core. By introducing
mixed-governance we can test if it will work for the providers before we
split them. It will take some time (and detailed AIP) to split, but in the
meantime we can see if we will be able to apply the mixed-governance after
the split. We will see if we can agree when it comes to expectations and
find solutions before we actually split.

J.

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Jarek Potiuk <ja...@potiuk.com>]

> +1 I agree this is a logical next step towards possibly separating
provider code from the Airflow code base (and it's useful even if we never
do that).

I am quite sure we will :)


On Thu, Jun 23, 2022 at 1:18 AM Oliveira, Niko <on...@amazon.com.invalid>
wrote:

> +1 I agree this is a logical next step towards possibly separating
> provider code from the Airflow code base (and it's useful even if we never
> do that).
>
> Cheers,
> Niko
> ------------------------------
> *From:* Kamil Breguła <dz...@gmail.com>
> *Sent:* Monday, June 20, 2022 1:30 PM
> *To:* dev@airflow.apache.org
> *Subject:* RE: [EXTERNAL][PROPOSAL] Provider's mixed governance model -
> first step of provider separation
>
>
> *CAUTION*: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
>
> I discussed this problem with Jarek. The group of stakeholders will be
> open and everyone can join, not just Google employees, etc. Release
> branches will be maintained in apache/airflow repository. Any non-committer
> change will still require PR. This means there is no vendor neutrality risk.
>
> +1 I think this is a good step forward.
>
> pon., 20 cze 2022 o 21:18 Jarek Potiuk <ja...@potiuk.com> napisał(a):
>
>> BTW. It will also be possible for anyone in the community to cherry-pick
>> changes from main and make a PR (which also a committer will have to
>> approve and merge). This is really no different that we have already done
>> with cherry-picked commits to "v1-10-stable" and "v-2-3-stable" branches by
>> non-committers. Random example here:
>> https://github.com/apache/airflow/pull/14090
>>
>> We do not give any privileges to the organisations. Quite the opposite -
>> we make them responsible for preparing the PRs to be reviewed by committers.
>>
>> J.
>>
>> On Mon, Jun 20, 2022 at 9:07 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>>
>>> > I think we should continue to be strictly vendor-neutral. No
>>> organization should be able to gain special privileges or control a
>>> project’s direction.
>>>
>>> This is strictly vendor-neutral - Kamil - we are going to release the
>>> same changes that we are releasing already in main providers, just
>>> selectively cherry-picked (and then reviewed and merged by committer to the
>>> branch in airflow repo) - why do you think it is non-vendor neutral?
>>>
>>> On Mon, Jun 20, 2022 at 9:04 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>>>
>>>> > We can keep these branches in forks managed by stakeholders teams,
>>>> but I am afraid of the benefit that it will be then copied by us to our
>>>> repository and then released by us. If the release was prepared by an
>>>> external team, I think we should make it clear that it was prepared by
>>>> another team, including by publishing on the Pypi account of the team that
>>>> dealt with it.
>>>>
>>>> > Yes this is exactly what I proposed.
>>>>
>>>> Correction. I misread it.
>>>>
>>>> We are going to merge - only the cherry-picked changes that have been
>>>> reviewed and merged by the committer. Same way as today. Just the process
>>>> of cherry-picks is going to be done by the stakeholders, selecting the
>>>> things to cherry-pick (all the changes to cherry-pick should be already
>>>> merged in main). What we are going to do is to release subset of the
>>>> changes we already approved (and released - because we are going to release
>>>> those changes in the latest provider). So there will be no "new" changes in
>>>> those forks - those will be just cherry-picked changes reviewed by the
>>>> committer. There is no reason to mark it as "other" code - it will be the
>>>> same changes that are going to be released anyway (just a subset of those).
>>>>
>>>> J.
>>>>
>>>> On Mon, Jun 20, 2022 at 9:00 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>>>>
>>>>> > Cherry-picking to branch v2-2* or 1.10.* can only be done by the
>>>>> committers, because only they have write permission to the apache/airflow
>>>>> repository. As far as I know, Github does not allow us to grant write-only
>>>>> permissions to the selected branch.
>>>>>
>>>>> Kamil - you misunderstood it. The branch will be in the FORK of those
>>>>> users's choice - not in airflow repo. committer will merge that branch in
>>>>> the same way we do as today - but with fast-forwarding rather than
>>>>> squashing.
>>>>>
>>>>> > We can keep these branches in forks managed by stakeholders teams,
>>>>> but I am afraid of the benefit that it will be then copied by us to our
>>>>> repository and then released by us. If the release was prepared by an
>>>>> external team, I think we should make it clear that it was prepared by
>>>>> another team, including by publishing on the Pypi account of the team that
>>>>> dealt with it.
>>>>>
>>>>> Yes this is exactly what I proposed.
>>>>>
>>>>> > I think that everything Apache PMC releases should be prepared and
>>>>> created fully within the apache / airflow repositories. If stakeholders
>>>>> team do not have such a possibility, we should figure out that these teams
>>>>> become part of the community, and therefore work together with the entire
>>>>> community, not in isolation. Only then will we be able to act in accordance
>>>>> with the Apache Way <https://www.apache.org/theapacheway/>, in
>>>>> particular each individual person will be able to contribute to the
>>>>> community as an individual, and not as a company or stakeholders team
>>>>> (Community of Peers) and no person will get special privileges just on the
>>>>> basis of their employment status (Earned Authority)
>>>>>
>>>>> Very true. And exactly follows my proposal :)
>>>>>
>>>>> J.
>>>>>
>>>>>
>>>>> On Mon, Jun 20, 2022 at 8:16 PM Kaxil Naik <ka...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> +1 -- We have discussed this during the Airflow Summit in-person with
>>>>>> Ash, Rafal (and his team), Jarek and I about this for a long time, and I
>>>>>> think this is a good step forward.
>>>>>>
>>>>>> Regards,
>>>>>> Kaxil
>>>>>>
>>>>>> On Mon, 20 Jun 2022 at 17:26, Kamil Breguła <dz...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I think we should continue to be strictly vendor-neutral. No
>>>>>>> organization should be able to gain special privileges or control a
>>>>>>> project’s direction.
>>>>>>>
>>>>>>> pon., 20 cze 2022 o 18:14 Kamil Breguła <dz...@gmail.com>
>>>>>>> napisał(a):
>>>>>>>
>>>>>>>> Cherry-picking to branch v2-2* or 1.10.* can only be done by the
>>>>>>>> committers, because only they have write permission to the apache/airflow
>>>>>>>> repository. As far as I know, Github does not allow us to grant write-only
>>>>>>>> permissions to the selected branch.
>>>>>>>>
>>>>>>>> We can keep these branches in forks managed by stakeholders teams,
>>>>>>>> but I am afraid of the benefit that it will be then copied by us to our
>>>>>>>> repository and then released by us. If the release was prepared by an
>>>>>>>> external team, I think we should make it clear that it was prepared by
>>>>>>>> another team, including by publishing on the Pypi account of the team that
>>>>>>>> dealt with it.
>>>>>>>>
>>>>>>>> I think that everything Apache PMC releases should be prepared and
>>>>>>>> created fully within the apache / airflow repositories. If stakeholders
>>>>>>>> team do not have such a possibility, we should figure out that these teams
>>>>>>>> become part of the community, and therefore work together with the entire
>>>>>>>> community, not in isolation. Only then will we be able to act in accordance
>>>>>>>> with the Apache Way <https://www.apache.org/theapacheway/>, in
>>>>>>>> particular each individual person will be able to contribute to the
>>>>>>>> community as an individual, and not as a company or stakeholders team
>>>>>>>> (Community of Peers) and no person will get special privileges just on the
>>>>>>>> basis of their employment status (Earned Authority)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> pon., 20 cze 2022 o 15:54 Jarek Potiuk <ja...@potiuk.com>
>>>>>>>> napisał(a):
>>>>>>>>
>>>>>>>>> > Will the people who maintain the providers' packages have the
>>>>>>>>> commiter
>>>>>>>>>
>>>>>>>>> Nope - similarly as we do in v2-2*  or what we did in 1.10.*
>>>>>>>>> cherry-picking can be done in separate branches (and in this case in
>>>>>>>>> forks).
>>>>>>>>> Then the branch can be fast-forwarded by the committer in the
>>>>>>>>> "airflow" repo. No problem with that.
>>>>>>>>>
>>>>>>>>> J.
>>>>>>>>>
>>>>>>>>> On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <dz...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Will the people who maintain the providers' packages have the
>>>>>>>>>> commiter
>>>>>>>>>> status? I guess it is necessary for people to have write access
>>>>>>>>>> to the
>>>>>>>>>> repository and therefore to be able to make cherry-pick changes
>>>>>>>>>> to the
>>>>>>>>>> branch.
>>>>>>>>>>
>>>>>>>>>> pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org>
>>>>>>>>>> napisał(a):
>>>>>>>>>> >
>>>>>>>>>> > +1
>>>>>>>>>> > From my side the proposal handles all concerns I raised in
>>>>>>>>>> previous threads.
>>>>>>>>>> > I think mixed-governance is a step in the right direction.
>>>>>>>>>> >
>>>>>>>>>> > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com>
>>>>>>>>>> wrote:
>>>>>>>>>> >>
>>>>>>>>>> >> Hello everyone,
>>>>>>>>>> >>
>>>>>>>>>> >> This is a follow-up after a few discussions started about
>>>>>>>>>> providers that were put on hold around  the summit. I held a number of
>>>>>>>>>> discussions during theSummit and after, and as result I think I have a
>>>>>>>>>> proposal that can move forward some of the "stalled" decisions we need to
>>>>>>>>>> make.
>>>>>>>>>> >>
>>>>>>>>>> >> TL;DR;
>>>>>>>>>> >>
>>>>>>>>>> >> My proposal is a "mixed-governance" model where "stakeholders"
>>>>>>>>>> are more responsible for cherry-picking and testing their providers
>>>>>>>>>> (including system testing) while Airflow PMC members will continue to be
>>>>>>>>>> responsible for releasing them.
>>>>>>>>>> >>
>>>>>>>>>> >> Why do we need that?
>>>>>>>>>> >>
>>>>>>>>>> >> Google, Amazon and possibly others teams who are interested in
>>>>>>>>>> maintaining more backwards compatible versions of their providers will
>>>>>>>>>> commit to make PRs of the cherry-picks for older release branches of their
>>>>>>>>>> providers. Those providers we release in parallel with the latest versions
>>>>>>>>>> during the normal provider cycle. We can deprecate changes more
>>>>>>>>>> aggressively in the "latest" release if we do that.
>>>>>>>>>> >>
>>>>>>>>>> >> Those cherry-picked PRs will be driven, tested and performed
>>>>>>>>>> by the stakeholder teams (Google/Amazon, Databricks, others) and will only
>>>>>>>>>> contain cherry-picks, while we - as PMC - will release them following the
>>>>>>>>>> ASF rules (this is very important for the ASF to follow strict release
>>>>>>>>>> policies regarding who and how performs releases).
>>>>>>>>>> >>
>>>>>>>>>> >> This also allows us to introduce similar rules for new
>>>>>>>>>> provider's acceptance for new providers for "main releases". It also allows
>>>>>>>>>> running the "system tests" for the provider under control of the
>>>>>>>>>> stakeholder (after applying AIP-47 changes).
>>>>>>>>>> >>
>>>>>>>>>> >> Example 1: Google team can cherry-pick changes to a
>>>>>>>>>> google-provider-6 branch and then we release a google 6.8.1 or 6.9.0
>>>>>>>>>> provider with some of the bug-fixes and features (together with - say
>>>>>>>>>> latest 8.1.0).
>>>>>>>>>> >>
>>>>>>>>>> >> Example 2: DataLake provider from Databricks - can get
>>>>>>>>>> accepted if Databricks commits to maintaining it. We will release the
>>>>>>>>>> provider as long as Databricks maintains it.
>>>>>>>>>> >>
>>>>>>>>>> >> Longer context:
>>>>>>>>>> >>
>>>>>>>>>> >> I have - in my mind so far - a longer roadmap for providers
>>>>>>>>>> that will lead them to be separated from the core and I want to write an
>>>>>>>>>> AIP about that soon. This AIP will detail all the steps needed - I will
>>>>>>>>>> work with multiple interested parties on it and it will take some time to
>>>>>>>>>> agree and complete.  But I want to start with something tangible that will
>>>>>>>>>> solve quite a few problems that were raised recently and something that
>>>>>>>>>> seems to be possible to be solved in the current provider release cycle
>>>>>>>>>> (till the end of June) and test some of the governance approach.
>>>>>>>>>> >>
>>>>>>>>>> >> This proposal simply builds on our semver approach - we do not
>>>>>>>>>> change it, we just start releasing some providers (those that have some
>>>>>>>>>> backing from stakeholders) in more than one version - including "latest"
>>>>>>>>>> and earlier, more backwards-compatible branches. Not all providers - just
>>>>>>>>>> some. Not all branches - just those that the stakeholders will commit to
>>>>>>>>>> maintain.
>>>>>>>>>> >>
>>>>>>>>>> >> We need such a commitment from stakeholders, because we - as
>>>>>>>>>> the  Airflow community and maintainers, want to only actively maintain the
>>>>>>>>>> latest releases, where it is in the interest of the stakeholders to
>>>>>>>>>> cherry-pick and test also earlier, more backwards compatible releases of
>>>>>>>>>> their choice.
>>>>>>>>>> >>
>>>>>>>>>> >> What problems this proposal solves:
>>>>>>>>>> >>
>>>>>>>>>> >> * Problem 1: DEPRECATION REMOVAL
>>>>>>>>>> >>
>>>>>>>>>> >> we can remove deprecations faster in "main" versions of the
>>>>>>>>>> providers - no need to introduce a deprecation policy - the stakeholders
>>>>>>>>>> for the providers will take care about cherry-picking and maintaining more
>>>>>>>>>> "backwards-compatible" versions. We are free to remove deprecations in
>>>>>>>>>> major releases (in a cherry-pickable way of course).
>>>>>>>>>> >>
>>>>>>>>>> >> * Problem 2: PROVIDERS DIVERGENCE
>>>>>>>>>> >>
>>>>>>>>>> >> We avoid the problem (already happened with the composer
>>>>>>>>>> release) that the stakeholders in a given provider had to release their own
>>>>>>>>>> version which was not available in their community - with some
>>>>>>>>>> cherry-picks. We want to avoid "diverging" there - by releasing the
>>>>>>>>>> cherry-picked providers by the community, we also give other users an
>>>>>>>>>> opportunity to follow "slower" deprecation policies for as long as it is
>>>>>>>>>> maintained.
>>>>>>>>>> >>
>>>>>>>>>> >> * Problem 3: PROVIDERS GOVERNANCE MODEL
>>>>>>>>>> >>
>>>>>>>>>> >> We are going to test a governance model that we might apply
>>>>>>>>>> when we split providers. We are talking about it for quite some time - but
>>>>>>>>>> this is what helps us to test the model where stakeholders provide more
>>>>>>>>>> "maintenance" while the community still takes care about releases. We (as
>>>>>>>>>> community) can commit to releasing such a version of a provider as long as
>>>>>>>>>> the stakeholder will actively maintain it. We can stop at any moment if we
>>>>>>>>>> do not have support from the stakeholder. If it works - we can keep it as a
>>>>>>>>>> long-term solution. In the future we can think of other scenarios (passing
>>>>>>>>>> ownership of a provider to stakeholders who want it - providing we want it
>>>>>>>>>> too) but we can decide about it when we learn from the mixed-governance
>>>>>>>>>> model and see if it works.
>>>>>>>>>> >>
>>>>>>>>>> >> * Problem 4: ACCEPTING NEW PROVIDERS
>>>>>>>>>> >>
>>>>>>>>>> >> If this is an acceptable approach - we can also apply a very
>>>>>>>>>> similar governance model to adding new providers and that should unblock
>>>>>>>>>> some of the PRs that are waiting for our decision. Knowing that we are
>>>>>>>>>> going to split and that we can expect "commitment" from a stakeholder, we
>>>>>>>>>> should be able to accept new providers. This might be possible assuming
>>>>>>>>>> that the stakeholder will make a similar commitment - but for new
>>>>>>>>>> providers, that commitment might also have to cover reviewing and testing
>>>>>>>>>> new changes. We might also decide as a community to stop releasing new
>>>>>>>>>> providers there if such support is missing. This way we can set the
>>>>>>>>>> expectations we have as a community for new providers - we will release
>>>>>>>>>> them as long as the stakeholder will actively make sure it is maintained.
>>>>>>>>>> >>
>>>>>>>>>> >> * Problem 5. SPLITTING PROVIDERS FROM CORE
>>>>>>>>>> >>
>>>>>>>>>> >> We all know we want to split providers from core. By
>>>>>>>>>> introducing mixed-governance we can test if it will work for the providers
>>>>>>>>>> before we split them. It will take some time (and detailed AIP) to split,
>>>>>>>>>> but in the meantime we can see if we will be able to apply the
>>>>>>>>>> mixed-governance after the split. We will see if we can agree when it comes
>>>>>>>>>> to expectations and find solutions before we actually split.
>>>>>>>>>> >>
>>>>>>>>>> >> J.
>>>>>>>>>>
>>>>>>>>>

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

["Oliveira, Niko" <on...@amazon.com.INVALID>]

+1 I agree this is a logical next step towards possibly separating provider code from the Airflow code base (and it's useful even if we never do that).

Cheers,
Niko

________________________________
From: Kamil Breguła <dz...@gmail.com>
Sent: Monday, June 20, 2022 1:30 PM
To: dev@airflow.apache.org
Subject: RE: [EXTERNAL][PROPOSAL] Provider's mixed governance model - first step of provider separation


CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.


I discussed this problem with Jarek. The group of stakeholders will be open and everyone can join, not just Google employees, etc. Release branches will be maintained in apache/airflow repository. Any non-committer change will still require PR. This means there is no vendor neutrality risk.

+1 I think this is a good step forward.

pon., 20 cze 2022 o 21:18 Jarek Potiuk <ja...@potiuk.com>> napisał(a):
BTW. It will also be possible for anyone in the community to cherry-pick changes from main and make a PR (which also a committer will have to approve and merge). This is really no different that we have already done with cherry-picked commits to "v1-10-stable" and "v-2-3-stable" branches by non-committers. Random example here:https://github.com/apache/airflow/pull/14090

We do not give any privileges to the organisations. Quite the opposite - we make them responsible for preparing the PRs to be reviewed by committers.

J.

On Mon, Jun 20, 2022 at 9:07 PM Jarek Potiuk <ja...@potiuk.com>> wrote:
> I think we should continue to be strictly vendor-neutral. No organization should be able to gain special privileges or control a project’s direction.

This is strictly vendor-neutral - Kamil - we are going to release the same changes that we are releasing already in main providers, just selectively cherry-picked (and then reviewed and merged by committer to the branch in airflow repo) - why do you think it is non-vendor neutral?

On Mon, Jun 20, 2022 at 9:04 PM Jarek Potiuk <ja...@potiuk.com>> wrote:
> We can keep these branches in forks managed by stakeholders teams, but I am afraid of the benefit that it will be then copied by us to our repository and then released by us. If the release was prepared by an external team, I think we should make it clear that it was prepared by another team, including by publishing on the Pypi account of the team that dealt with it.

> Yes this is exactly what I proposed.

Correction. I misread it.

We are going to merge - only the cherry-picked changes that have been reviewed and merged by the committer. Same way as today. Just the process of cherry-picks is going to be done by the stakeholders, selecting the things to cherry-pick (all the changes to cherry-pick should be already merged in main). What we are going to do is to release subset of the changes we already approved (and released - because we are going to release those changes in the latest provider). So there will be no "new" changes in those forks - those will be just cherry-picked changes reviewed by the committer. There is no reason to mark it as "other" code - it will be the same changes that are going to be released anyway (just a subset of those).

J.

On Mon, Jun 20, 2022 at 9:00 PM Jarek Potiuk <ja...@potiuk.com>> wrote:
> Cherry-picking to branch v2-2* or 1.10.* can only be done by the committers, because only they have write permission to the apache/airflow repository. As far as I know, Github does not allow us to grant write-only permissions to the selected branch.

Kamil - you misunderstood it. The branch will be in the FORK of those users's choice - not in airflow repo. committer will merge that branch in the same way we do as today - but with fast-forwarding rather than squashing.

> We can keep these branches in forks managed by stakeholders teams, but I am afraid of the benefit that it will be then copied by us to our repository and then released by us. If the release was prepared by an external team, I think we should make it clear that it was prepared by another team, including by publishing on the Pypi account of the team that dealt with it.

Yes this is exactly what I proposed.

> I think that everything Apache PMC releases should be prepared and created fully within the apache / airflow repositories. If stakeholders team do not have such a possibility, we should figure out that these teams become part of the community, and therefore work together with the entire community, not in isolation. Only then will we be able to act in accordance with the Apache Way<https://www.apache.org/theapacheway/>, in particular each individual person will be able to contribute to the community as an individual, and not as a company or stakeholders team (Community of Peers) and no person will get special privileges just on the basis of their employment status (Earned Authority)

Very true. And exactly follows my proposal :)

J.


On Mon, Jun 20, 2022 at 8:16 PM Kaxil Naik <ka...@gmail.com>> wrote:
+1 -- We have discussed this during the Airflow Summit in-person with Ash, Rafal (and his team), Jarek and I about this for a long time, and I think this is a good step forward.

Regards,
Kaxil

On Mon, 20 Jun 2022 at 17:26, Kamil Breguła <dz...@gmail.com>> wrote:
I think we should continue to be strictly vendor-neutral. No organization should be able to gain special privileges or control a project’s direction.

pon., 20 cze 2022 o 18:14 Kamil Breguła <dz...@gmail.com>> napisał(a):
Cherry-picking to branch v2-2* or 1.10.* can only be done by the committers, because only they have write permission to the apache/airflow repository. As far as I know, Github does not allow us to grant write-only permissions to the selected branch.

We can keep these branches in forks managed by stakeholders teams, but I am afraid of the benefit that it will be then copied by us to our repository and then released by us. If the release was prepared by an external team, I think we should make it clear that it was prepared by another team, including by publishing on the Pypi account of the team that dealt with it.

I think that everything Apache PMC releases should be prepared and created fully within the apache / airflow repositories. If stakeholders team do not have such a possibility, we should figure out that these teams become part of the community, and therefore work together with the entire community, not in isolation. Only then will we be able to act in accordance with the Apache Way<https://www.apache.org/theapacheway/>, in particular each individual person will be able to contribute to the community as an individual, and not as a company or stakeholders team (Community of Peers) and no person will get special privileges just on the basis of their employment status (Earned Authority)



pon., 20 cze 2022 o 15:54 Jarek Potiuk <ja...@potiuk.com>> napisał(a):
> Will the people who maintain the providers' packages have the commiter

Nope - similarly as we do in v2-2*  or what we did in 1.10.* cherry-picking can be done in separate branches (and in this case in forks).
Then the branch can be fast-forwarded by the committer in the "airflow" repo. No problem with that.

J.

On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <dz...@gmail.com>> wrote:
Will the people who maintain the providers' packages have the commiter
status? I guess it is necessary for people to have write access to the
repository and therefore to be able to make cherry-pick changes to the
branch.

pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org>> napisał(a):
>
> +1
> From my side the proposal handles all concerns I raised in previous threads.
> I think mixed-governance is a step in the right direction.
>
> On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com>> wrote:
>>
>> Hello everyone,
>>
>> This is a follow-up after a few discussions started about providers that were put on hold around  the summit. I held a number of discussions during theSummit and after, and as result I think I have a proposal that can move forward some of the "stalled" decisions we need to make.
>>
>> TL;DR;
>>
>> My proposal is a "mixed-governance" model where "stakeholders" are more responsible for cherry-picking and testing their providers (including system testing) while Airflow PMC members will continue to be responsible for releasing them.
>>
>> Why do we need that?
>>
>> Google, Amazon and possibly others teams who are interested in maintaining more backwards compatible versions of their providers will commit to make PRs of the cherry-picks for older release branches of their providers. Those providers we release in parallel with the latest versions during the normal provider cycle. We can deprecate changes more aggressively in the "latest" release if we do that.
>>
>> Those cherry-picked PRs will be driven, tested and performed by the stakeholder teams (Google/Amazon, Databricks, others) and will only contain cherry-picks, while we - as PMC - will release them following the ASF rules (this is very important for the ASF to follow strict release policies regarding who and how performs releases).
>>
>> This also allows us to introduce similar rules for new provider's acceptance for new providers for "main releases". It also allows running the "system tests" for the provider under control of the stakeholder (after applying AIP-47 changes).
>>
>> Example 1: Google team can cherry-pick changes to a google-provider-6 branch and then we release a google 6.8.1 or 6.9.0 provider with some of the bug-fixes and features (together with - say latest 8.1.0).
>>
>> Example 2: DataLake provider from Databricks - can get accepted if Databricks commits to maintaining it. We will release the provider as long as Databricks maintains it.
>>
>> Longer context:
>>
>> I have - in my mind so far - a longer roadmap for providers that will lead them to be separated from the core and I want to write an AIP about that soon. This AIP will detail all the steps needed - I will work with multiple interested parties on it and it will take some time to agree and complete.  But I want to start with something tangible that will solve quite a few problems that were raised recently and something that seems to be possible to be solved in the current provider release cycle (till the end of June) and test some of the governance approach.
>>
>> This proposal simply builds on our semver approach - we do not change it, we just start releasing some providers (those that have some backing from stakeholders) in more than one version - including "latest" and earlier, more backwards-compatible branches. Not all providers - just some. Not all branches - just those that the stakeholders will commit to maintain.
>>
>> We need such a commitment from stakeholders, because we - as the  Airflow community and maintainers, want to only actively maintain the latest releases, where it is in the interest of the stakeholders to cherry-pick and test also earlier, more backwards compatible releases of their choice.
>>
>> What problems this proposal solves:
>>
>> * Problem 1: DEPRECATION REMOVAL
>>
>> we can remove deprecations faster in "main" versions of the providers - no need to introduce a deprecation policy - the stakeholders for the providers will take care about cherry-picking and maintaining more "backwards-compatible" versions. We are free to remove deprecations in major releases (in a cherry-pickable way of course).
>>
>> * Problem 2: PROVIDERS DIVERGENCE
>>
>> We avoid the problem (already happened with the composer release) that the stakeholders in a given provider had to release their own version which was not available in their community - with some cherry-picks. We want to avoid "diverging" there - by releasing the cherry-picked providers by the community, we also give other users an opportunity to follow "slower" deprecation policies for as long as it is maintained.
>>
>> * Problem 3: PROVIDERS GOVERNANCE MODEL
>>
>> We are going to test a governance model that we might apply when we split providers. We are talking about it for quite some time - but this is what helps us to test the model where stakeholders provide more "maintenance" while the community still takes care about releases. We (as community) can commit to releasing such a version of a provider as long as the stakeholder will actively maintain it. We can stop at any moment if we do not have support from the stakeholder. If it works - we can keep it as a long-term solution. In the future we can think of other scenarios (passing ownership of a provider to stakeholders who want it - providing we want it too) but we can decide about it when we learn from the mixed-governance model and see if it works.
>>
>> * Problem 4: ACCEPTING NEW PROVIDERS
>>
>> If this is an acceptable approach - we can also apply a very similar governance model to adding new providers and that should unblock some of the PRs that are waiting for our decision. Knowing that we are going to split and that we can expect "commitment" from a stakeholder, we should be able to accept new providers. This might be possible assuming that the stakeholder will make a similar commitment - but for new providers, that commitment might also have to cover reviewing and testing new changes. We might also decide as a community to stop releasing new providers there if such support is missing. This way we can set the expectations we have as a community for new providers - we will release them as long as the stakeholder will actively make sure it is maintained.
>>
>> * Problem 5. SPLITTING PROVIDERS FROM CORE
>>
>> We all know we want to split providers from core. By introducing mixed-governance we can test if it will work for the providers before we split them. It will take some time (and detailed AIP) to split, but in the meantime we can see if we will be able to apply the mixed-governance after the split. We will see if we can agree when it comes to expectations and find solutions before we actually split.
>>
>> J.

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Kamil Breguła <dz...@gmail.com>]

I discussed this problem with Jarek. The group of stakeholders will be open
and everyone can join, not just Google employees, etc. Release branches
will be maintained in apache/airflow repository. Any non-committer change
will still require PR. This means there is no vendor neutrality risk.

+1 I think this is a good step forward.

pon., 20 cze 2022 o 21:18 Jarek Potiuk <ja...@potiuk.com> napisał(a):

> BTW. It will also be possible for anyone in the community to cherry-pick
> changes from main and make a PR (which also a committer will have to
> approve and merge). This is really no different that we have already done
> with cherry-picked commits to "v1-10-stable" and "v-2-3-stable" branches by
> non-committers. Random example here:
> https://github.com/apache/airflow/pull/14090
>
> We do not give any privileges to the organisations. Quite the opposite -
> we make them responsible for preparing the PRs to be reviewed by committers.
>
> J.
>
> On Mon, Jun 20, 2022 at 9:07 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
>> > I think we should continue to be strictly vendor-neutral. No
>> organization should be able to gain special privileges or control a
>> project’s direction.
>>
>> This is strictly vendor-neutral - Kamil - we are going to release the
>> same changes that we are releasing already in main providers, just
>> selectively cherry-picked (and then reviewed and merged by committer to the
>> branch in airflow repo) - why do you think it is non-vendor neutral?
>>
>> On Mon, Jun 20, 2022 at 9:04 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>>
>>> > We can keep these branches in forks managed by stakeholders teams, but
>>> I am afraid of the benefit that it will be then copied by us to our
>>> repository and then released by us. If the release was prepared by an
>>> external team, I think we should make it clear that it was prepared by
>>> another team, including by publishing on the Pypi account of the team that
>>> dealt with it.
>>>
>>> > Yes this is exactly what I proposed.
>>>
>>> Correction. I misread it.
>>>
>>> We are going to merge - only the cherry-picked changes that have been
>>> reviewed and merged by the committer. Same way as today. Just the process
>>> of cherry-picks is going to be done by the stakeholders, selecting the
>>> things to cherry-pick (all the changes to cherry-pick should be already
>>> merged in main). What we are going to do is to release subset of the
>>> changes we already approved (and released - because we are going to release
>>> those changes in the latest provider). So there will be no "new" changes in
>>> those forks - those will be just cherry-picked changes reviewed by the
>>> committer. There is no reason to mark it as "other" code - it will be the
>>> same changes that are going to be released anyway (just a subset of those).
>>>
>>> J.
>>>
>>> On Mon, Jun 20, 2022 at 9:00 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>>>
>>>> > Cherry-picking to branch v2-2* or 1.10.* can only be done by the
>>>> committers, because only they have write permission to the apache/airflow
>>>> repository. As far as I know, Github does not allow us to grant write-only
>>>> permissions to the selected branch.
>>>>
>>>> Kamil - you misunderstood it. The branch will be in the FORK of those
>>>> users's choice - not in airflow repo. committer will merge that branch in
>>>> the same way we do as today - but with fast-forwarding rather than
>>>> squashing.
>>>>
>>>> > We can keep these branches in forks managed by stakeholders teams,
>>>> but I am afraid of the benefit that it will be then copied by us to our
>>>> repository and then released by us. If the release was prepared by an
>>>> external team, I think we should make it clear that it was prepared by
>>>> another team, including by publishing on the Pypi account of the team that
>>>> dealt with it.
>>>>
>>>> Yes this is exactly what I proposed.
>>>>
>>>> > I think that everything Apache PMC releases should be prepared and
>>>> created fully within the apache / airflow repositories. If stakeholders
>>>> team do not have such a possibility, we should figure out that these teams
>>>> become part of the community, and therefore work together with the entire
>>>> community, not in isolation. Only then will we be able to act in accordance
>>>> with the Apache Way <https://www.apache.org/theapacheway/>, in
>>>> particular each individual person will be able to contribute to the
>>>> community as an individual, and not as a company or stakeholders team
>>>> (Community of Peers) and no person will get special privileges just on the
>>>> basis of their employment status (Earned Authority)
>>>>
>>>> Very true. And exactly follows my proposal :)
>>>>
>>>> J.
>>>>
>>>>
>>>> On Mon, Jun 20, 2022 at 8:16 PM Kaxil Naik <ka...@gmail.com> wrote:
>>>>
>>>>> +1 -- We have discussed this during the Airflow Summit in-person with
>>>>> Ash, Rafal (and his team), Jarek and I about this for a long time, and I
>>>>> think this is a good step forward.
>>>>>
>>>>> Regards,
>>>>> Kaxil
>>>>>
>>>>> On Mon, 20 Jun 2022 at 17:26, Kamil Breguła <dz...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I think we should continue to be strictly vendor-neutral. No
>>>>>> organization should be able to gain special privileges or control a
>>>>>> project’s direction.
>>>>>>
>>>>>> pon., 20 cze 2022 o 18:14 Kamil Breguła <dz...@gmail.com>
>>>>>> napisał(a):
>>>>>>
>>>>>>> Cherry-picking to branch v2-2* or 1.10.* can only be done by the
>>>>>>> committers, because only they have write permission to the apache/airflow
>>>>>>> repository. As far as I know, Github does not allow us to grant write-only
>>>>>>> permissions to the selected branch.
>>>>>>>
>>>>>>> We can keep these branches in forks managed by stakeholders teams,
>>>>>>> but I am afraid of the benefit that it will be then copied by us to our
>>>>>>> repository and then released by us. If the release was prepared by an
>>>>>>> external team, I think we should make it clear that it was prepared by
>>>>>>> another team, including by publishing on the Pypi account of the team that
>>>>>>> dealt with it.
>>>>>>>
>>>>>>> I think that everything Apache PMC releases should be prepared and
>>>>>>> created fully within the apache / airflow repositories. If stakeholders
>>>>>>> team do not have such a possibility, we should figure out that these teams
>>>>>>> become part of the community, and therefore work together with the entire
>>>>>>> community, not in isolation. Only then will we be able to act in accordance
>>>>>>> with the Apache Way <https://www.apache.org/theapacheway/>, in
>>>>>>> particular each individual person will be able to contribute to the
>>>>>>> community as an individual, and not as a company or stakeholders team
>>>>>>> (Community of Peers) and no person will get special privileges just on the
>>>>>>> basis of their employment status (Earned Authority)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> pon., 20 cze 2022 o 15:54 Jarek Potiuk <ja...@potiuk.com>
>>>>>>> napisał(a):
>>>>>>>
>>>>>>>> > Will the people who maintain the providers' packages have the
>>>>>>>> commiter
>>>>>>>>
>>>>>>>> Nope - similarly as we do in v2-2*  or what we did in 1.10.*
>>>>>>>> cherry-picking can be done in separate branches (and in this case in
>>>>>>>> forks).
>>>>>>>> Then the branch can be fast-forwarded by the committer in the
>>>>>>>> "airflow" repo. No problem with that.
>>>>>>>>
>>>>>>>> J.
>>>>>>>>
>>>>>>>> On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <dz...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Will the people who maintain the providers' packages have the
>>>>>>>>> commiter
>>>>>>>>> status? I guess it is necessary for people to have write access to
>>>>>>>>> the
>>>>>>>>> repository and therefore to be able to make cherry-pick changes to
>>>>>>>>> the
>>>>>>>>> branch.
>>>>>>>>>
>>>>>>>>> pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org>
>>>>>>>>> napisał(a):
>>>>>>>>> >
>>>>>>>>> > +1
>>>>>>>>> > From my side the proposal handles all concerns I raised in
>>>>>>>>> previous threads.
>>>>>>>>> > I think mixed-governance is a step in the right direction.
>>>>>>>>> >
>>>>>>>>> > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com>
>>>>>>>>> wrote:
>>>>>>>>> >>
>>>>>>>>> >> Hello everyone,
>>>>>>>>> >>
>>>>>>>>> >> This is a follow-up after a few discussions started about
>>>>>>>>> providers that were put on hold around  the summit. I held a number of
>>>>>>>>> discussions during theSummit and after, and as result I think I have a
>>>>>>>>> proposal that can move forward some of the "stalled" decisions we need to
>>>>>>>>> make.
>>>>>>>>> >>
>>>>>>>>> >> TL;DR;
>>>>>>>>> >>
>>>>>>>>> >> My proposal is a "mixed-governance" model where "stakeholders"
>>>>>>>>> are more responsible for cherry-picking and testing their providers
>>>>>>>>> (including system testing) while Airflow PMC members will continue to be
>>>>>>>>> responsible for releasing them.
>>>>>>>>> >>
>>>>>>>>> >> Why do we need that?
>>>>>>>>> >>
>>>>>>>>> >> Google, Amazon and possibly others teams who are interested in
>>>>>>>>> maintaining more backwards compatible versions of their providers will
>>>>>>>>> commit to make PRs of the cherry-picks for older release branches of their
>>>>>>>>> providers. Those providers we release in parallel with the latest versions
>>>>>>>>> during the normal provider cycle. We can deprecate changes more
>>>>>>>>> aggressively in the "latest" release if we do that.
>>>>>>>>> >>
>>>>>>>>> >> Those cherry-picked PRs will be driven, tested and performed by
>>>>>>>>> the stakeholder teams (Google/Amazon, Databricks, others) and will only
>>>>>>>>> contain cherry-picks, while we - as PMC - will release them following the
>>>>>>>>> ASF rules (this is very important for the ASF to follow strict release
>>>>>>>>> policies regarding who and how performs releases).
>>>>>>>>> >>
>>>>>>>>> >> This also allows us to introduce similar rules for new
>>>>>>>>> provider's acceptance for new providers for "main releases". It also allows
>>>>>>>>> running the "system tests" for the provider under control of the
>>>>>>>>> stakeholder (after applying AIP-47 changes).
>>>>>>>>> >>
>>>>>>>>> >> Example 1: Google team can cherry-pick changes to a
>>>>>>>>> google-provider-6 branch and then we release a google 6.8.1 or 6.9.0
>>>>>>>>> provider with some of the bug-fixes and features (together with - say
>>>>>>>>> latest 8.1.0).
>>>>>>>>> >>
>>>>>>>>> >> Example 2: DataLake provider from Databricks - can get accepted
>>>>>>>>> if Databricks commits to maintaining it. We will release the provider as
>>>>>>>>> long as Databricks maintains it.
>>>>>>>>> >>
>>>>>>>>> >> Longer context:
>>>>>>>>> >>
>>>>>>>>> >> I have - in my mind so far - a longer roadmap for providers
>>>>>>>>> that will lead them to be separated from the core and I want to write an
>>>>>>>>> AIP about that soon. This AIP will detail all the steps needed - I will
>>>>>>>>> work with multiple interested parties on it and it will take some time to
>>>>>>>>> agree and complete.  But I want to start with something tangible that will
>>>>>>>>> solve quite a few problems that were raised recently and something that
>>>>>>>>> seems to be possible to be solved in the current provider release cycle
>>>>>>>>> (till the end of June) and test some of the governance approach.
>>>>>>>>> >>
>>>>>>>>> >> This proposal simply builds on our semver approach - we do not
>>>>>>>>> change it, we just start releasing some providers (those that have some
>>>>>>>>> backing from stakeholders) in more than one version - including "latest"
>>>>>>>>> and earlier, more backwards-compatible branches. Not all providers - just
>>>>>>>>> some. Not all branches - just those that the stakeholders will commit to
>>>>>>>>> maintain.
>>>>>>>>> >>
>>>>>>>>> >> We need such a commitment from stakeholders, because we - as
>>>>>>>>> the  Airflow community and maintainers, want to only actively maintain the
>>>>>>>>> latest releases, where it is in the interest of the stakeholders to
>>>>>>>>> cherry-pick and test also earlier, more backwards compatible releases of
>>>>>>>>> their choice.
>>>>>>>>> >>
>>>>>>>>> >> What problems this proposal solves:
>>>>>>>>> >>
>>>>>>>>> >> * Problem 1: DEPRECATION REMOVAL
>>>>>>>>> >>
>>>>>>>>> >> we can remove deprecations faster in "main" versions of the
>>>>>>>>> providers - no need to introduce a deprecation policy - the stakeholders
>>>>>>>>> for the providers will take care about cherry-picking and maintaining more
>>>>>>>>> "backwards-compatible" versions. We are free to remove deprecations in
>>>>>>>>> major releases (in a cherry-pickable way of course).
>>>>>>>>> >>
>>>>>>>>> >> * Problem 2: PROVIDERS DIVERGENCE
>>>>>>>>> >>
>>>>>>>>> >> We avoid the problem (already happened with the composer
>>>>>>>>> release) that the stakeholders in a given provider had to release their own
>>>>>>>>> version which was not available in their community - with some
>>>>>>>>> cherry-picks. We want to avoid "diverging" there - by releasing the
>>>>>>>>> cherry-picked providers by the community, we also give other users an
>>>>>>>>> opportunity to follow "slower" deprecation policies for as long as it is
>>>>>>>>> maintained.
>>>>>>>>> >>
>>>>>>>>> >> * Problem 3: PROVIDERS GOVERNANCE MODEL
>>>>>>>>> >>
>>>>>>>>> >> We are going to test a governance model that we might apply
>>>>>>>>> when we split providers. We are talking about it for quite some time - but
>>>>>>>>> this is what helps us to test the model where stakeholders provide more
>>>>>>>>> "maintenance" while the community still takes care about releases. We (as
>>>>>>>>> community) can commit to releasing such a version of a provider as long as
>>>>>>>>> the stakeholder will actively maintain it. We can stop at any moment if we
>>>>>>>>> do not have support from the stakeholder. If it works - we can keep it as a
>>>>>>>>> long-term solution. In the future we can think of other scenarios (passing
>>>>>>>>> ownership of a provider to stakeholders who want it - providing we want it
>>>>>>>>> too) but we can decide about it when we learn from the mixed-governance
>>>>>>>>> model and see if it works.
>>>>>>>>> >>
>>>>>>>>> >> * Problem 4: ACCEPTING NEW PROVIDERS
>>>>>>>>> >>
>>>>>>>>> >> If this is an acceptable approach - we can also apply a very
>>>>>>>>> similar governance model to adding new providers and that should unblock
>>>>>>>>> some of the PRs that are waiting for our decision. Knowing that we are
>>>>>>>>> going to split and that we can expect "commitment" from a stakeholder, we
>>>>>>>>> should be able to accept new providers. This might be possible assuming
>>>>>>>>> that the stakeholder will make a similar commitment - but for new
>>>>>>>>> providers, that commitment might also have to cover reviewing and testing
>>>>>>>>> new changes. We might also decide as a community to stop releasing new
>>>>>>>>> providers there if such support is missing. This way we can set the
>>>>>>>>> expectations we have as a community for new providers - we will release
>>>>>>>>> them as long as the stakeholder will actively make sure it is maintained.
>>>>>>>>> >>
>>>>>>>>> >> * Problem 5. SPLITTING PROVIDERS FROM CORE
>>>>>>>>> >>
>>>>>>>>> >> We all know we want to split providers from core. By
>>>>>>>>> introducing mixed-governance we can test if it will work for the providers
>>>>>>>>> before we split them. It will take some time (and detailed AIP) to split,
>>>>>>>>> but in the meantime we can see if we will be able to apply the
>>>>>>>>> mixed-governance after the split. We will see if we can agree when it comes
>>>>>>>>> to expectations and find solutions before we actually split.
>>>>>>>>> >>
>>>>>>>>> >> J.
>>>>>>>>>
>>>>>>>>

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Jarek Potiuk <ja...@potiuk.com>]

BTW. It will also be possible for anyone in the community to cherry-pick
changes from main and make a PR (which also a committer will have to
approve and merge). This is really no different that we have already done
with cherry-picked commits to "v1-10-stable" and "v-2-3-stable" branches by
non-committers. Random example here:
https://github.com/apache/airflow/pull/14090

We do not give any privileges to the organisations. Quite the opposite - we
make them responsible for preparing the PRs to be reviewed by committers.

J.

On Mon, Jun 20, 2022 at 9:07 PM Jarek Potiuk <ja...@potiuk.com> wrote:

> > I think we should continue to be strictly vendor-neutral. No
> organization should be able to gain special privileges or control a
> project’s direction.
>
> This is strictly vendor-neutral - Kamil - we are going to release the same
> changes that we are releasing already in main providers, just selectively
> cherry-picked (and then reviewed and merged by committer to the branch in
> airflow repo) - why do you think it is non-vendor neutral?
>
> On Mon, Jun 20, 2022 at 9:04 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
>> > We can keep these branches in forks managed by stakeholders teams, but
>> I am afraid of the benefit that it will be then copied by us to our
>> repository and then released by us. If the release was prepared by an
>> external team, I think we should make it clear that it was prepared by
>> another team, including by publishing on the Pypi account of the team that
>> dealt with it.
>>
>> > Yes this is exactly what I proposed.
>>
>> Correction. I misread it.
>>
>> We are going to merge - only the cherry-picked changes that have been
>> reviewed and merged by the committer. Same way as today. Just the process
>> of cherry-picks is going to be done by the stakeholders, selecting the
>> things to cherry-pick (all the changes to cherry-pick should be already
>> merged in main). What we are going to do is to release subset of the
>> changes we already approved (and released - because we are going to release
>> those changes in the latest provider). So there will be no "new" changes in
>> those forks - those will be just cherry-picked changes reviewed by the
>> committer. There is no reason to mark it as "other" code - it will be the
>> same changes that are going to be released anyway (just a subset of those).
>>
>> J.
>>
>> On Mon, Jun 20, 2022 at 9:00 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>>
>>> > Cherry-picking to branch v2-2* or 1.10.* can only be done by the
>>> committers, because only they have write permission to the apache/airflow
>>> repository. As far as I know, Github does not allow us to grant write-only
>>> permissions to the selected branch.
>>>
>>> Kamil - you misunderstood it. The branch will be in the FORK of those
>>> users's choice - not in airflow repo. committer will merge that branch in
>>> the same way we do as today - but with fast-forwarding rather than
>>> squashing.
>>>
>>> > We can keep these branches in forks managed by stakeholders teams, but
>>> I am afraid of the benefit that it will be then copied by us to our
>>> repository and then released by us. If the release was prepared by an
>>> external team, I think we should make it clear that it was prepared by
>>> another team, including by publishing on the Pypi account of the team that
>>> dealt with it.
>>>
>>> Yes this is exactly what I proposed.
>>>
>>> > I think that everything Apache PMC releases should be prepared and
>>> created fully within the apache / airflow repositories. If stakeholders
>>> team do not have such a possibility, we should figure out that these teams
>>> become part of the community, and therefore work together with the entire
>>> community, not in isolation. Only then will we be able to act in accordance
>>> with the Apache Way <https://www.apache.org/theapacheway/>, in
>>> particular each individual person will be able to contribute to the
>>> community as an individual, and not as a company or stakeholders team
>>> (Community of Peers) and no person will get special privileges just on the
>>> basis of their employment status (Earned Authority)
>>>
>>> Very true. And exactly follows my proposal :)
>>>
>>> J.
>>>
>>>
>>> On Mon, Jun 20, 2022 at 8:16 PM Kaxil Naik <ka...@gmail.com> wrote:
>>>
>>>> +1 -- We have discussed this during the Airflow Summit in-person with
>>>> Ash, Rafal (and his team), Jarek and I about this for a long time, and I
>>>> think this is a good step forward.
>>>>
>>>> Regards,
>>>> Kaxil
>>>>
>>>> On Mon, 20 Jun 2022 at 17:26, Kamil Breguła <dz...@gmail.com> wrote:
>>>>
>>>>> I think we should continue to be strictly vendor-neutral. No
>>>>> organization should be able to gain special privileges or control a
>>>>> project’s direction.
>>>>>
>>>>> pon., 20 cze 2022 o 18:14 Kamil Breguła <dz...@gmail.com>
>>>>> napisał(a):
>>>>>
>>>>>> Cherry-picking to branch v2-2* or 1.10.* can only be done by the
>>>>>> committers, because only they have write permission to the apache/airflow
>>>>>> repository. As far as I know, Github does not allow us to grant write-only
>>>>>> permissions to the selected branch.
>>>>>>
>>>>>> We can keep these branches in forks managed by stakeholders teams,
>>>>>> but I am afraid of the benefit that it will be then copied by us to our
>>>>>> repository and then released by us. If the release was prepared by an
>>>>>> external team, I think we should make it clear that it was prepared by
>>>>>> another team, including by publishing on the Pypi account of the team that
>>>>>> dealt with it.
>>>>>>
>>>>>> I think that everything Apache PMC releases should be prepared and
>>>>>> created fully within the apache / airflow repositories. If stakeholders
>>>>>> team do not have such a possibility, we should figure out that these teams
>>>>>> become part of the community, and therefore work together with the entire
>>>>>> community, not in isolation. Only then will we be able to act in accordance
>>>>>> with the Apache Way <https://www.apache.org/theapacheway/>, in
>>>>>> particular each individual person will be able to contribute to the
>>>>>> community as an individual, and not as a company or stakeholders team
>>>>>> (Community of Peers) and no person will get special privileges just on the
>>>>>> basis of their employment status (Earned Authority)
>>>>>>
>>>>>>
>>>>>>
>>>>>> pon., 20 cze 2022 o 15:54 Jarek Potiuk <ja...@potiuk.com> napisał(a):
>>>>>>
>>>>>>> > Will the people who maintain the providers' packages have the
>>>>>>> commiter
>>>>>>>
>>>>>>> Nope - similarly as we do in v2-2*  or what we did in 1.10.*
>>>>>>> cherry-picking can be done in separate branches (and in this case in
>>>>>>> forks).
>>>>>>> Then the branch can be fast-forwarded by the committer in the
>>>>>>> "airflow" repo. No problem with that.
>>>>>>>
>>>>>>> J.
>>>>>>>
>>>>>>> On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <dz...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Will the people who maintain the providers' packages have the
>>>>>>>> commiter
>>>>>>>> status? I guess it is necessary for people to have write access to
>>>>>>>> the
>>>>>>>> repository and therefore to be able to make cherry-pick changes to
>>>>>>>> the
>>>>>>>> branch.
>>>>>>>>
>>>>>>>> pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org>
>>>>>>>> napisał(a):
>>>>>>>> >
>>>>>>>> > +1
>>>>>>>> > From my side the proposal handles all concerns I raised in
>>>>>>>> previous threads.
>>>>>>>> > I think mixed-governance is a step in the right direction.
>>>>>>>> >
>>>>>>>> > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com>
>>>>>>>> wrote:
>>>>>>>> >>
>>>>>>>> >> Hello everyone,
>>>>>>>> >>
>>>>>>>> >> This is a follow-up after a few discussions started about
>>>>>>>> providers that were put on hold around  the summit. I held a number of
>>>>>>>> discussions during theSummit and after, and as result I think I have a
>>>>>>>> proposal that can move forward some of the "stalled" decisions we need to
>>>>>>>> make.
>>>>>>>> >>
>>>>>>>> >> TL;DR;
>>>>>>>> >>
>>>>>>>> >> My proposal is a "mixed-governance" model where "stakeholders"
>>>>>>>> are more responsible for cherry-picking and testing their providers
>>>>>>>> (including system testing) while Airflow PMC members will continue to be
>>>>>>>> responsible for releasing them.
>>>>>>>> >>
>>>>>>>> >> Why do we need that?
>>>>>>>> >>
>>>>>>>> >> Google, Amazon and possibly others teams who are interested in
>>>>>>>> maintaining more backwards compatible versions of their providers will
>>>>>>>> commit to make PRs of the cherry-picks for older release branches of their
>>>>>>>> providers. Those providers we release in parallel with the latest versions
>>>>>>>> during the normal provider cycle. We can deprecate changes more
>>>>>>>> aggressively in the "latest" release if we do that.
>>>>>>>> >>
>>>>>>>> >> Those cherry-picked PRs will be driven, tested and performed by
>>>>>>>> the stakeholder teams (Google/Amazon, Databricks, others) and will only
>>>>>>>> contain cherry-picks, while we - as PMC - will release them following the
>>>>>>>> ASF rules (this is very important for the ASF to follow strict release
>>>>>>>> policies regarding who and how performs releases).
>>>>>>>> >>
>>>>>>>> >> This also allows us to introduce similar rules for new
>>>>>>>> provider's acceptance for new providers for "main releases". It also allows
>>>>>>>> running the "system tests" for the provider under control of the
>>>>>>>> stakeholder (after applying AIP-47 changes).
>>>>>>>> >>
>>>>>>>> >> Example 1: Google team can cherry-pick changes to a
>>>>>>>> google-provider-6 branch and then we release a google 6.8.1 or 6.9.0
>>>>>>>> provider with some of the bug-fixes and features (together with - say
>>>>>>>> latest 8.1.0).
>>>>>>>> >>
>>>>>>>> >> Example 2: DataLake provider from Databricks - can get accepted
>>>>>>>> if Databricks commits to maintaining it. We will release the provider as
>>>>>>>> long as Databricks maintains it.
>>>>>>>> >>
>>>>>>>> >> Longer context:
>>>>>>>> >>
>>>>>>>> >> I have - in my mind so far - a longer roadmap for providers that
>>>>>>>> will lead them to be separated from the core and I want to write an AIP
>>>>>>>> about that soon. This AIP will detail all the steps needed - I will work
>>>>>>>> with multiple interested parties on it and it will take some time to agree
>>>>>>>> and complete.  But I want to start with something tangible that will solve
>>>>>>>> quite a few problems that were raised recently and something that seems to
>>>>>>>> be possible to be solved in the current provider release cycle (till the
>>>>>>>> end of June) and test some of the governance approach.
>>>>>>>> >>
>>>>>>>> >> This proposal simply builds on our semver approach - we do not
>>>>>>>> change it, we just start releasing some providers (those that have some
>>>>>>>> backing from stakeholders) in more than one version - including "latest"
>>>>>>>> and earlier, more backwards-compatible branches. Not all providers - just
>>>>>>>> some. Not all branches - just those that the stakeholders will commit to
>>>>>>>> maintain.
>>>>>>>> >>
>>>>>>>> >> We need such a commitment from stakeholders, because we - as
>>>>>>>> the  Airflow community and maintainers, want to only actively maintain the
>>>>>>>> latest releases, where it is in the interest of the stakeholders to
>>>>>>>> cherry-pick and test also earlier, more backwards compatible releases of
>>>>>>>> their choice.
>>>>>>>> >>
>>>>>>>> >> What problems this proposal solves:
>>>>>>>> >>
>>>>>>>> >> * Problem 1: DEPRECATION REMOVAL
>>>>>>>> >>
>>>>>>>> >> we can remove deprecations faster in "main" versions of the
>>>>>>>> providers - no need to introduce a deprecation policy - the stakeholders
>>>>>>>> for the providers will take care about cherry-picking and maintaining more
>>>>>>>> "backwards-compatible" versions. We are free to remove deprecations in
>>>>>>>> major releases (in a cherry-pickable way of course).
>>>>>>>> >>
>>>>>>>> >> * Problem 2: PROVIDERS DIVERGENCE
>>>>>>>> >>
>>>>>>>> >> We avoid the problem (already happened with the composer
>>>>>>>> release) that the stakeholders in a given provider had to release their own
>>>>>>>> version which was not available in their community - with some
>>>>>>>> cherry-picks. We want to avoid "diverging" there - by releasing the
>>>>>>>> cherry-picked providers by the community, we also give other users an
>>>>>>>> opportunity to follow "slower" deprecation policies for as long as it is
>>>>>>>> maintained.
>>>>>>>> >>
>>>>>>>> >> * Problem 3: PROVIDERS GOVERNANCE MODEL
>>>>>>>> >>
>>>>>>>> >> We are going to test a governance model that we might apply when
>>>>>>>> we split providers. We are talking about it for quite some time - but this
>>>>>>>> is what helps us to test the model where stakeholders provide more
>>>>>>>> "maintenance" while the community still takes care about releases. We (as
>>>>>>>> community) can commit to releasing such a version of a provider as long as
>>>>>>>> the stakeholder will actively maintain it. We can stop at any moment if we
>>>>>>>> do not have support from the stakeholder. If it works - we can keep it as a
>>>>>>>> long-term solution. In the future we can think of other scenarios (passing
>>>>>>>> ownership of a provider to stakeholders who want it - providing we want it
>>>>>>>> too) but we can decide about it when we learn from the mixed-governance
>>>>>>>> model and see if it works.
>>>>>>>> >>
>>>>>>>> >> * Problem 4: ACCEPTING NEW PROVIDERS
>>>>>>>> >>
>>>>>>>> >> If this is an acceptable approach - we can also apply a very
>>>>>>>> similar governance model to adding new providers and that should unblock
>>>>>>>> some of the PRs that are waiting for our decision. Knowing that we are
>>>>>>>> going to split and that we can expect "commitment" from a stakeholder, we
>>>>>>>> should be able to accept new providers. This might be possible assuming
>>>>>>>> that the stakeholder will make a similar commitment - but for new
>>>>>>>> providers, that commitment might also have to cover reviewing and testing
>>>>>>>> new changes. We might also decide as a community to stop releasing new
>>>>>>>> providers there if such support is missing. This way we can set the
>>>>>>>> expectations we have as a community for new providers - we will release
>>>>>>>> them as long as the stakeholder will actively make sure it is maintained.
>>>>>>>> >>
>>>>>>>> >> * Problem 5. SPLITTING PROVIDERS FROM CORE
>>>>>>>> >>
>>>>>>>> >> We all know we want to split providers from core. By introducing
>>>>>>>> mixed-governance we can test if it will work for the providers before we
>>>>>>>> split them. It will take some time (and detailed AIP) to split, but in the
>>>>>>>> meantime we can see if we will be able to apply the mixed-governance after
>>>>>>>> the split. We will see if we can agree when it comes to expectations and
>>>>>>>> find solutions before we actually split.
>>>>>>>> >>
>>>>>>>> >> J.
>>>>>>>>
>>>>>>>

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Jarek Potiuk <ja...@potiuk.com>]

> I think we should continue to be strictly vendor-neutral. No organization
should be able to gain special privileges or control a project’s direction.

This is strictly vendor-neutral - Kamil - we are going to release the same
changes that we are releasing already in main providers, just selectively
cherry-picked (and then reviewed and merged by committer to the branch in
airflow repo) - why do you think it is non-vendor neutral?

On Mon, Jun 20, 2022 at 9:04 PM Jarek Potiuk <ja...@potiuk.com> wrote:

> > We can keep these branches in forks managed by stakeholders teams, but I
> am afraid of the benefit that it will be then copied by us to our
> repository and then released by us. If the release was prepared by an
> external team, I think we should make it clear that it was prepared by
> another team, including by publishing on the Pypi account of the team that
> dealt with it.
>
> > Yes this is exactly what I proposed.
>
> Correction. I misread it.
>
> We are going to merge - only the cherry-picked changes that have been
> reviewed and merged by the committer. Same way as today. Just the process
> of cherry-picks is going to be done by the stakeholders, selecting the
> things to cherry-pick (all the changes to cherry-pick should be already
> merged in main). What we are going to do is to release subset of the
> changes we already approved (and released - because we are going to release
> those changes in the latest provider). So there will be no "new" changes in
> those forks - those will be just cherry-picked changes reviewed by the
> committer. There is no reason to mark it as "other" code - it will be the
> same changes that are going to be released anyway (just a subset of those).
>
> J.
>
> On Mon, Jun 20, 2022 at 9:00 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
>> > Cherry-picking to branch v2-2* or 1.10.* can only be done by the
>> committers, because only they have write permission to the apache/airflow
>> repository. As far as I know, Github does not allow us to grant write-only
>> permissions to the selected branch.
>>
>> Kamil - you misunderstood it. The branch will be in the FORK of those
>> users's choice - not in airflow repo. committer will merge that branch in
>> the same way we do as today - but with fast-forwarding rather than
>> squashing.
>>
>> > We can keep these branches in forks managed by stakeholders teams, but
>> I am afraid of the benefit that it will be then copied by us to our
>> repository and then released by us. If the release was prepared by an
>> external team, I think we should make it clear that it was prepared by
>> another team, including by publishing on the Pypi account of the team that
>> dealt with it.
>>
>> Yes this is exactly what I proposed.
>>
>> > I think that everything Apache PMC releases should be prepared and
>> created fully within the apache / airflow repositories. If stakeholders
>> team do not have such a possibility, we should figure out that these teams
>> become part of the community, and therefore work together with the entire
>> community, not in isolation. Only then will we be able to act in accordance
>> with the Apache Way <https://www.apache.org/theapacheway/>, in
>> particular each individual person will be able to contribute to the
>> community as an individual, and not as a company or stakeholders team
>> (Community of Peers) and no person will get special privileges just on the
>> basis of their employment status (Earned Authority)
>>
>> Very true. And exactly follows my proposal :)
>>
>> J.
>>
>>
>> On Mon, Jun 20, 2022 at 8:16 PM Kaxil Naik <ka...@gmail.com> wrote:
>>
>>> +1 -- We have discussed this during the Airflow Summit in-person with
>>> Ash, Rafal (and his team), Jarek and I about this for a long time, and I
>>> think this is a good step forward.
>>>
>>> Regards,
>>> Kaxil
>>>
>>> On Mon, 20 Jun 2022 at 17:26, Kamil Breguła <dz...@gmail.com> wrote:
>>>
>>>> I think we should continue to be strictly vendor-neutral. No
>>>> organization should be able to gain special privileges or control a
>>>> project’s direction.
>>>>
>>>> pon., 20 cze 2022 o 18:14 Kamil Breguła <dz...@gmail.com>
>>>> napisał(a):
>>>>
>>>>> Cherry-picking to branch v2-2* or 1.10.* can only be done by the
>>>>> committers, because only they have write permission to the apache/airflow
>>>>> repository. As far as I know, Github does not allow us to grant write-only
>>>>> permissions to the selected branch.
>>>>>
>>>>> We can keep these branches in forks managed by stakeholders teams, but
>>>>> I am afraid of the benefit that it will be then copied by us to our
>>>>> repository and then released by us. If the release was prepared by an
>>>>> external team, I think we should make it clear that it was prepared by
>>>>> another team, including by publishing on the Pypi account of the team that
>>>>> dealt with it.
>>>>>
>>>>> I think that everything Apache PMC releases should be prepared and
>>>>> created fully within the apache / airflow repositories. If stakeholders
>>>>> team do not have such a possibility, we should figure out that these teams
>>>>> become part of the community, and therefore work together with the entire
>>>>> community, not in isolation. Only then will we be able to act in accordance
>>>>> with the Apache Way <https://www.apache.org/theapacheway/>, in
>>>>> particular each individual person will be able to contribute to the
>>>>> community as an individual, and not as a company or stakeholders team
>>>>> (Community of Peers) and no person will get special privileges just on the
>>>>> basis of their employment status (Earned Authority)
>>>>>
>>>>>
>>>>>
>>>>> pon., 20 cze 2022 o 15:54 Jarek Potiuk <ja...@potiuk.com> napisał(a):
>>>>>
>>>>>> > Will the people who maintain the providers' packages have the
>>>>>> commiter
>>>>>>
>>>>>> Nope - similarly as we do in v2-2*  or what we did in 1.10.*
>>>>>> cherry-picking can be done in separate branches (and in this case in
>>>>>> forks).
>>>>>> Then the branch can be fast-forwarded by the committer in the
>>>>>> "airflow" repo. No problem with that.
>>>>>>
>>>>>> J.
>>>>>>
>>>>>> On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <dz...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Will the people who maintain the providers' packages have the
>>>>>>> commiter
>>>>>>> status? I guess it is necessary for people to have write access to
>>>>>>> the
>>>>>>> repository and therefore to be able to make cherry-pick changes to
>>>>>>> the
>>>>>>> branch.
>>>>>>>
>>>>>>> pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org>
>>>>>>> napisał(a):
>>>>>>> >
>>>>>>> > +1
>>>>>>> > From my side the proposal handles all concerns I raised in
>>>>>>> previous threads.
>>>>>>> > I think mixed-governance is a step in the right direction.
>>>>>>> >
>>>>>>> > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com>
>>>>>>> wrote:
>>>>>>> >>
>>>>>>> >> Hello everyone,
>>>>>>> >>
>>>>>>> >> This is a follow-up after a few discussions started about
>>>>>>> providers that were put on hold around  the summit. I held a number of
>>>>>>> discussions during theSummit and after, and as result I think I have a
>>>>>>> proposal that can move forward some of the "stalled" decisions we need to
>>>>>>> make.
>>>>>>> >>
>>>>>>> >> TL;DR;
>>>>>>> >>
>>>>>>> >> My proposal is a "mixed-governance" model where "stakeholders"
>>>>>>> are more responsible for cherry-picking and testing their providers
>>>>>>> (including system testing) while Airflow PMC members will continue to be
>>>>>>> responsible for releasing them.
>>>>>>> >>
>>>>>>> >> Why do we need that?
>>>>>>> >>
>>>>>>> >> Google, Amazon and possibly others teams who are interested in
>>>>>>> maintaining more backwards compatible versions of their providers will
>>>>>>> commit to make PRs of the cherry-picks for older release branches of their
>>>>>>> providers. Those providers we release in parallel with the latest versions
>>>>>>> during the normal provider cycle. We can deprecate changes more
>>>>>>> aggressively in the "latest" release if we do that.
>>>>>>> >>
>>>>>>> >> Those cherry-picked PRs will be driven, tested and performed by
>>>>>>> the stakeholder teams (Google/Amazon, Databricks, others) and will only
>>>>>>> contain cherry-picks, while we - as PMC - will release them following the
>>>>>>> ASF rules (this is very important for the ASF to follow strict release
>>>>>>> policies regarding who and how performs releases).
>>>>>>> >>
>>>>>>> >> This also allows us to introduce similar rules for new provider's
>>>>>>> acceptance for new providers for "main releases". It also allows running
>>>>>>> the "system tests" for the provider under control of the stakeholder (after
>>>>>>> applying AIP-47 changes).
>>>>>>> >>
>>>>>>> >> Example 1: Google team can cherry-pick changes to a
>>>>>>> google-provider-6 branch and then we release a google 6.8.1 or 6.9.0
>>>>>>> provider with some of the bug-fixes and features (together with - say
>>>>>>> latest 8.1.0).
>>>>>>> >>
>>>>>>> >> Example 2: DataLake provider from Databricks - can get accepted
>>>>>>> if Databricks commits to maintaining it. We will release the provider as
>>>>>>> long as Databricks maintains it.
>>>>>>> >>
>>>>>>> >> Longer context:
>>>>>>> >>
>>>>>>> >> I have - in my mind so far - a longer roadmap for providers that
>>>>>>> will lead them to be separated from the core and I want to write an AIP
>>>>>>> about that soon. This AIP will detail all the steps needed - I will work
>>>>>>> with multiple interested parties on it and it will take some time to agree
>>>>>>> and complete.  But I want to start with something tangible that will solve
>>>>>>> quite a few problems that were raised recently and something that seems to
>>>>>>> be possible to be solved in the current provider release cycle (till the
>>>>>>> end of June) and test some of the governance approach.
>>>>>>> >>
>>>>>>> >> This proposal simply builds on our semver approach - we do not
>>>>>>> change it, we just start releasing some providers (those that have some
>>>>>>> backing from stakeholders) in more than one version - including "latest"
>>>>>>> and earlier, more backwards-compatible branches. Not all providers - just
>>>>>>> some. Not all branches - just those that the stakeholders will commit to
>>>>>>> maintain.
>>>>>>> >>
>>>>>>> >> We need such a commitment from stakeholders, because we - as the
>>>>>>> Airflow community and maintainers, want to only actively maintain the
>>>>>>> latest releases, where it is in the interest of the stakeholders to
>>>>>>> cherry-pick and test also earlier, more backwards compatible releases of
>>>>>>> their choice.
>>>>>>> >>
>>>>>>> >> What problems this proposal solves:
>>>>>>> >>
>>>>>>> >> * Problem 1: DEPRECATION REMOVAL
>>>>>>> >>
>>>>>>> >> we can remove deprecations faster in "main" versions of the
>>>>>>> providers - no need to introduce a deprecation policy - the stakeholders
>>>>>>> for the providers will take care about cherry-picking and maintaining more
>>>>>>> "backwards-compatible" versions. We are free to remove deprecations in
>>>>>>> major releases (in a cherry-pickable way of course).
>>>>>>> >>
>>>>>>> >> * Problem 2: PROVIDERS DIVERGENCE
>>>>>>> >>
>>>>>>> >> We avoid the problem (already happened with the composer release)
>>>>>>> that the stakeholders in a given provider had to release their own version
>>>>>>> which was not available in their community - with some cherry-picks. We
>>>>>>> want to avoid "diverging" there - by releasing the cherry-picked providers
>>>>>>> by the community, we also give other users an opportunity to follow
>>>>>>> "slower" deprecation policies for as long as it is maintained.
>>>>>>> >>
>>>>>>> >> * Problem 3: PROVIDERS GOVERNANCE MODEL
>>>>>>> >>
>>>>>>> >> We are going to test a governance model that we might apply when
>>>>>>> we split providers. We are talking about it for quite some time - but this
>>>>>>> is what helps us to test the model where stakeholders provide more
>>>>>>> "maintenance" while the community still takes care about releases. We (as
>>>>>>> community) can commit to releasing such a version of a provider as long as
>>>>>>> the stakeholder will actively maintain it. We can stop at any moment if we
>>>>>>> do not have support from the stakeholder. If it works - we can keep it as a
>>>>>>> long-term solution. In the future we can think of other scenarios (passing
>>>>>>> ownership of a provider to stakeholders who want it - providing we want it
>>>>>>> too) but we can decide about it when we learn from the mixed-governance
>>>>>>> model and see if it works.
>>>>>>> >>
>>>>>>> >> * Problem 4: ACCEPTING NEW PROVIDERS
>>>>>>> >>
>>>>>>> >> If this is an acceptable approach - we can also apply a very
>>>>>>> similar governance model to adding new providers and that should unblock
>>>>>>> some of the PRs that are waiting for our decision. Knowing that we are
>>>>>>> going to split and that we can expect "commitment" from a stakeholder, we
>>>>>>> should be able to accept new providers. This might be possible assuming
>>>>>>> that the stakeholder will make a similar commitment - but for new
>>>>>>> providers, that commitment might also have to cover reviewing and testing
>>>>>>> new changes. We might also decide as a community to stop releasing new
>>>>>>> providers there if such support is missing. This way we can set the
>>>>>>> expectations we have as a community for new providers - we will release
>>>>>>> them as long as the stakeholder will actively make sure it is maintained.
>>>>>>> >>
>>>>>>> >> * Problem 5. SPLITTING PROVIDERS FROM CORE
>>>>>>> >>
>>>>>>> >> We all know we want to split providers from core. By introducing
>>>>>>> mixed-governance we can test if it will work for the providers before we
>>>>>>> split them. It will take some time (and detailed AIP) to split, but in the
>>>>>>> meantime we can see if we will be able to apply the mixed-governance after
>>>>>>> the split. We will see if we can agree when it comes to expectations and
>>>>>>> find solutions before we actually split.
>>>>>>> >>
>>>>>>> >> J.
>>>>>>>
>>>>>>

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Jarek Potiuk <ja...@potiuk.com>]

> We can keep these branches in forks managed by stakeholders teams, but I
am afraid of the benefit that it will be then copied by us to our
repository and then released by us. If the release was prepared by an
external team, I think we should make it clear that it was prepared by
another team, including by publishing on the Pypi account of the team that
dealt with it.

> Yes this is exactly what I proposed.

Correction. I misread it.

We are going to merge - only the cherry-picked changes that have been
reviewed and merged by the committer. Same way as today. Just the process
of cherry-picks is going to be done by the stakeholders, selecting the
things to cherry-pick (all the changes to cherry-pick should be already
merged in main). What we are going to do is to release subset of the
changes we already approved (and released - because we are going to release
those changes in the latest provider). So there will be no "new" changes in
those forks - those will be just cherry-picked changes reviewed by the
committer. There is no reason to mark it as "other" code - it will be the
same changes that are going to be released anyway (just a subset of those).

J.

On Mon, Jun 20, 2022 at 9:00 PM Jarek Potiuk <ja...@potiuk.com> wrote:

> > Cherry-picking to branch v2-2* or 1.10.* can only be done by the
> committers, because only they have write permission to the apache/airflow
> repository. As far as I know, Github does not allow us to grant write-only
> permissions to the selected branch.
>
> Kamil - you misunderstood it. The branch will be in the FORK of those
> users's choice - not in airflow repo. committer will merge that branch in
> the same way we do as today - but with fast-forwarding rather than
> squashing.
>
> > We can keep these branches in forks managed by stakeholders teams, but I
> am afraid of the benefit that it will be then copied by us to our
> repository and then released by us. If the release was prepared by an
> external team, I think we should make it clear that it was prepared by
> another team, including by publishing on the Pypi account of the team that
> dealt with it.
>
> Yes this is exactly what I proposed.
>
> > I think that everything Apache PMC releases should be prepared and
> created fully within the apache / airflow repositories. If stakeholders
> team do not have such a possibility, we should figure out that these teams
> become part of the community, and therefore work together with the entire
> community, not in isolation. Only then will we be able to act in accordance
> with the Apache Way <https://www.apache.org/theapacheway/>, in particular
> each individual person will be able to contribute to the community as an
> individual, and not as a company or stakeholders team (Community of Peers)
> and no person will get special privileges just on the basis of their
> employment status (Earned Authority)
>
> Very true. And exactly follows my proposal :)
>
> J.
>
>
> On Mon, Jun 20, 2022 at 8:16 PM Kaxil Naik <ka...@gmail.com> wrote:
>
>> +1 -- We have discussed this during the Airflow Summit in-person with
>> Ash, Rafal (and his team), Jarek and I about this for a long time, and I
>> think this is a good step forward.
>>
>> Regards,
>> Kaxil
>>
>> On Mon, 20 Jun 2022 at 17:26, Kamil Breguła <dz...@gmail.com> wrote:
>>
>>> I think we should continue to be strictly vendor-neutral. No
>>> organization should be able to gain special privileges or control a
>>> project’s direction.
>>>
>>> pon., 20 cze 2022 o 18:14 Kamil Breguła <dz...@gmail.com> napisał(a):
>>>
>>>> Cherry-picking to branch v2-2* or 1.10.* can only be done by the
>>>> committers, because only they have write permission to the apache/airflow
>>>> repository. As far as I know, Github does not allow us to grant write-only
>>>> permissions to the selected branch.
>>>>
>>>> We can keep these branches in forks managed by stakeholders teams, but
>>>> I am afraid of the benefit that it will be then copied by us to our
>>>> repository and then released by us. If the release was prepared by an
>>>> external team, I think we should make it clear that it was prepared by
>>>> another team, including by publishing on the Pypi account of the team that
>>>> dealt with it.
>>>>
>>>> I think that everything Apache PMC releases should be prepared and
>>>> created fully within the apache / airflow repositories. If stakeholders
>>>> team do not have such a possibility, we should figure out that these teams
>>>> become part of the community, and therefore work together with the entire
>>>> community, not in isolation. Only then will we be able to act in accordance
>>>> with the Apache Way <https://www.apache.org/theapacheway/>, in
>>>> particular each individual person will be able to contribute to the
>>>> community as an individual, and not as a company or stakeholders team
>>>> (Community of Peers) and no person will get special privileges just on the
>>>> basis of their employment status (Earned Authority)
>>>>
>>>>
>>>>
>>>> pon., 20 cze 2022 o 15:54 Jarek Potiuk <ja...@potiuk.com> napisał(a):
>>>>
>>>>> > Will the people who maintain the providers' packages have the
>>>>> commiter
>>>>>
>>>>> Nope - similarly as we do in v2-2*  or what we did in 1.10.*
>>>>> cherry-picking can be done in separate branches (and in this case in
>>>>> forks).
>>>>> Then the branch can be fast-forwarded by the committer in the
>>>>> "airflow" repo. No problem with that.
>>>>>
>>>>> J.
>>>>>
>>>>> On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <dz...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Will the people who maintain the providers' packages have the commiter
>>>>>> status? I guess it is necessary for people to have write access to the
>>>>>> repository and therefore to be able to make cherry-pick changes to the
>>>>>> branch.
>>>>>>
>>>>>> pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org> napisał(a):
>>>>>> >
>>>>>> > +1
>>>>>> > From my side the proposal handles all concerns I raised in previous
>>>>>> threads.
>>>>>> > I think mixed-governance is a step in the right direction.
>>>>>> >
>>>>>> > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> Hello everyone,
>>>>>> >>
>>>>>> >> This is a follow-up after a few discussions started about
>>>>>> providers that were put on hold around  the summit. I held a number of
>>>>>> discussions during theSummit and after, and as result I think I have a
>>>>>> proposal that can move forward some of the "stalled" decisions we need to
>>>>>> make.
>>>>>> >>
>>>>>> >> TL;DR;
>>>>>> >>
>>>>>> >> My proposal is a "mixed-governance" model where "stakeholders" are
>>>>>> more responsible for cherry-picking and testing their providers (including
>>>>>> system testing) while Airflow PMC members will continue to be responsible
>>>>>> for releasing them.
>>>>>> >>
>>>>>> >> Why do we need that?
>>>>>> >>
>>>>>> >> Google, Amazon and possibly others teams who are interested in
>>>>>> maintaining more backwards compatible versions of their providers will
>>>>>> commit to make PRs of the cherry-picks for older release branches of their
>>>>>> providers. Those providers we release in parallel with the latest versions
>>>>>> during the normal provider cycle. We can deprecate changes more
>>>>>> aggressively in the "latest" release if we do that.
>>>>>> >>
>>>>>> >> Those cherry-picked PRs will be driven, tested and performed by
>>>>>> the stakeholder teams (Google/Amazon, Databricks, others) and will only
>>>>>> contain cherry-picks, while we - as PMC - will release them following the
>>>>>> ASF rules (this is very important for the ASF to follow strict release
>>>>>> policies regarding who and how performs releases).
>>>>>> >>
>>>>>> >> This also allows us to introduce similar rules for new provider's
>>>>>> acceptance for new providers for "main releases". It also allows running
>>>>>> the "system tests" for the provider under control of the stakeholder (after
>>>>>> applying AIP-47 changes).
>>>>>> >>
>>>>>> >> Example 1: Google team can cherry-pick changes to a
>>>>>> google-provider-6 branch and then we release a google 6.8.1 or 6.9.0
>>>>>> provider with some of the bug-fixes and features (together with - say
>>>>>> latest 8.1.0).
>>>>>> >>
>>>>>> >> Example 2: DataLake provider from Databricks - can get accepted if
>>>>>> Databricks commits to maintaining it. We will release the provider as long
>>>>>> as Databricks maintains it.
>>>>>> >>
>>>>>> >> Longer context:
>>>>>> >>
>>>>>> >> I have - in my mind so far - a longer roadmap for providers that
>>>>>> will lead them to be separated from the core and I want to write an AIP
>>>>>> about that soon. This AIP will detail all the steps needed - I will work
>>>>>> with multiple interested parties on it and it will take some time to agree
>>>>>> and complete.  But I want to start with something tangible that will solve
>>>>>> quite a few problems that were raised recently and something that seems to
>>>>>> be possible to be solved in the current provider release cycle (till the
>>>>>> end of June) and test some of the governance approach.
>>>>>> >>
>>>>>> >> This proposal simply builds on our semver approach - we do not
>>>>>> change it, we just start releasing some providers (those that have some
>>>>>> backing from stakeholders) in more than one version - including "latest"
>>>>>> and earlier, more backwards-compatible branches. Not all providers - just
>>>>>> some. Not all branches - just those that the stakeholders will commit to
>>>>>> maintain.
>>>>>> >>
>>>>>> >> We need such a commitment from stakeholders, because we - as the
>>>>>> Airflow community and maintainers, want to only actively maintain the
>>>>>> latest releases, where it is in the interest of the stakeholders to
>>>>>> cherry-pick and test also earlier, more backwards compatible releases of
>>>>>> their choice.
>>>>>> >>
>>>>>> >> What problems this proposal solves:
>>>>>> >>
>>>>>> >> * Problem 1: DEPRECATION REMOVAL
>>>>>> >>
>>>>>> >> we can remove deprecations faster in "main" versions of the
>>>>>> providers - no need to introduce a deprecation policy - the stakeholders
>>>>>> for the providers will take care about cherry-picking and maintaining more
>>>>>> "backwards-compatible" versions. We are free to remove deprecations in
>>>>>> major releases (in a cherry-pickable way of course).
>>>>>> >>
>>>>>> >> * Problem 2: PROVIDERS DIVERGENCE
>>>>>> >>
>>>>>> >> We avoid the problem (already happened with the composer release)
>>>>>> that the stakeholders in a given provider had to release their own version
>>>>>> which was not available in their community - with some cherry-picks. We
>>>>>> want to avoid "diverging" there - by releasing the cherry-picked providers
>>>>>> by the community, we also give other users an opportunity to follow
>>>>>> "slower" deprecation policies for as long as it is maintained.
>>>>>> >>
>>>>>> >> * Problem 3: PROVIDERS GOVERNANCE MODEL
>>>>>> >>
>>>>>> >> We are going to test a governance model that we might apply when
>>>>>> we split providers. We are talking about it for quite some time - but this
>>>>>> is what helps us to test the model where stakeholders provide more
>>>>>> "maintenance" while the community still takes care about releases. We (as
>>>>>> community) can commit to releasing such a version of a provider as long as
>>>>>> the stakeholder will actively maintain it. We can stop at any moment if we
>>>>>> do not have support from the stakeholder. If it works - we can keep it as a
>>>>>> long-term solution. In the future we can think of other scenarios (passing
>>>>>> ownership of a provider to stakeholders who want it - providing we want it
>>>>>> too) but we can decide about it when we learn from the mixed-governance
>>>>>> model and see if it works.
>>>>>> >>
>>>>>> >> * Problem 4: ACCEPTING NEW PROVIDERS
>>>>>> >>
>>>>>> >> If this is an acceptable approach - we can also apply a very
>>>>>> similar governance model to adding new providers and that should unblock
>>>>>> some of the PRs that are waiting for our decision. Knowing that we are
>>>>>> going to split and that we can expect "commitment" from a stakeholder, we
>>>>>> should be able to accept new providers. This might be possible assuming
>>>>>> that the stakeholder will make a similar commitment - but for new
>>>>>> providers, that commitment might also have to cover reviewing and testing
>>>>>> new changes. We might also decide as a community to stop releasing new
>>>>>> providers there if such support is missing. This way we can set the
>>>>>> expectations we have as a community for new providers - we will release
>>>>>> them as long as the stakeholder will actively make sure it is maintained.
>>>>>> >>
>>>>>> >> * Problem 5. SPLITTING PROVIDERS FROM CORE
>>>>>> >>
>>>>>> >> We all know we want to split providers from core. By introducing
>>>>>> mixed-governance we can test if it will work for the providers before we
>>>>>> split them. It will take some time (and detailed AIP) to split, but in the
>>>>>> meantime we can see if we will be able to apply the mixed-governance after
>>>>>> the split. We will see if we can agree when it comes to expectations and
>>>>>> find solutions before we actually split.
>>>>>> >>
>>>>>> >> J.
>>>>>>
>>>>>

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Jarek Potiuk <ja...@potiuk.com>]

> Cherry-picking to branch v2-2* or 1.10.* can only be done by the
committers, because only they have write permission to the apache/airflow
repository. As far as I know, Github does not allow us to grant write-only
permissions to the selected branch.

Kamil - you misunderstood it. The branch will be in the FORK of those
users's choice - not in airflow repo. committer will merge that branch in
the same way we do as today - but with fast-forwarding rather than
squashing.

> We can keep these branches in forks managed by stakeholders teams, but I
am afraid of the benefit that it will be then copied by us to our
repository and then released by us. If the release was prepared by an
external team, I think we should make it clear that it was prepared by
another team, including by publishing on the Pypi account of the team that
dealt with it.

Yes this is exactly what I proposed.

> I think that everything Apache PMC releases should be prepared and
created fully within the apache / airflow repositories. If stakeholders
team do not have such a possibility, we should figure out that these teams
become part of the community, and therefore work together with the entire
community, not in isolation. Only then will we be able to act in accordance
with the Apache Way <https://www.apache.org/theapacheway/>, in particular
each individual person will be able to contribute to the community as an
individual, and not as a company or stakeholders team (Community of Peers)
and no person will get special privileges just on the basis of their
employment status (Earned Authority)

Very true. And exactly follows my proposal :)

J.


On Mon, Jun 20, 2022 at 8:16 PM Kaxil Naik <ka...@gmail.com> wrote:

> +1 -- We have discussed this during the Airflow Summit in-person with Ash,
> Rafal (and his team), Jarek and I about this for a long time, and I think
> this is a good step forward.
>
> Regards,
> Kaxil
>
> On Mon, 20 Jun 2022 at 17:26, Kamil Breguła <dz...@gmail.com> wrote:
>
>> I think we should continue to be strictly vendor-neutral. No
>> organization should be able to gain special privileges or control a
>> project’s direction.
>>
>> pon., 20 cze 2022 o 18:14 Kamil Breguła <dz...@gmail.com> napisał(a):
>>
>>> Cherry-picking to branch v2-2* or 1.10.* can only be done by the
>>> committers, because only they have write permission to the apache/airflow
>>> repository. As far as I know, Github does not allow us to grant write-only
>>> permissions to the selected branch.
>>>
>>> We can keep these branches in forks managed by stakeholders teams, but I
>>> am afraid of the benefit that it will be then copied by us to our
>>> repository and then released by us. If the release was prepared by an
>>> external team, I think we should make it clear that it was prepared by
>>> another team, including by publishing on the Pypi account of the team that
>>> dealt with it.
>>>
>>> I think that everything Apache PMC releases should be prepared and
>>> created fully within the apache / airflow repositories. If stakeholders
>>> team do not have such a possibility, we should figure out that these teams
>>> become part of the community, and therefore work together with the entire
>>> community, not in isolation. Only then will we be able to act in accordance
>>> with the Apache Way <https://www.apache.org/theapacheway/>, in
>>> particular each individual person will be able to contribute to the
>>> community as an individual, and not as a company or stakeholders team
>>> (Community of Peers) and no person will get special privileges just on the
>>> basis of their employment status (Earned Authority)
>>>
>>>
>>>
>>> pon., 20 cze 2022 o 15:54 Jarek Potiuk <ja...@potiuk.com> napisał(a):
>>>
>>>> > Will the people who maintain the providers' packages have the commiter
>>>>
>>>> Nope - similarly as we do in v2-2*  or what we did in 1.10.*
>>>> cherry-picking can be done in separate branches (and in this case in
>>>> forks).
>>>> Then the branch can be fast-forwarded by the committer in the "airflow"
>>>> repo. No problem with that.
>>>>
>>>> J.
>>>>
>>>> On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <dz...@gmail.com>
>>>> wrote:
>>>>
>>>>> Will the people who maintain the providers' packages have the commiter
>>>>> status? I guess it is necessary for people to have write access to the
>>>>> repository and therefore to be able to make cherry-pick changes to the
>>>>> branch.
>>>>>
>>>>> pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org> napisał(a):
>>>>> >
>>>>> > +1
>>>>> > From my side the proposal handles all concerns I raised in previous
>>>>> threads.
>>>>> > I think mixed-governance is a step in the right direction.
>>>>> >
>>>>> > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com>
>>>>> wrote:
>>>>> >>
>>>>> >> Hello everyone,
>>>>> >>
>>>>> >> This is a follow-up after a few discussions started about providers
>>>>> that were put on hold around  the summit. I held a number of discussions
>>>>> during theSummit and after, and as result I think I have a proposal that
>>>>> can move forward some of the "stalled" decisions we need to make.
>>>>> >>
>>>>> >> TL;DR;
>>>>> >>
>>>>> >> My proposal is a "mixed-governance" model where "stakeholders" are
>>>>> more responsible for cherry-picking and testing their providers (including
>>>>> system testing) while Airflow PMC members will continue to be responsible
>>>>> for releasing them.
>>>>> >>
>>>>> >> Why do we need that?
>>>>> >>
>>>>> >> Google, Amazon and possibly others teams who are interested in
>>>>> maintaining more backwards compatible versions of their providers will
>>>>> commit to make PRs of the cherry-picks for older release branches of their
>>>>> providers. Those providers we release in parallel with the latest versions
>>>>> during the normal provider cycle. We can deprecate changes more
>>>>> aggressively in the "latest" release if we do that.
>>>>> >>
>>>>> >> Those cherry-picked PRs will be driven, tested and performed by the
>>>>> stakeholder teams (Google/Amazon, Databricks, others) and will only contain
>>>>> cherry-picks, while we - as PMC - will release them following the ASF rules
>>>>> (this is very important for the ASF to follow strict release policies
>>>>> regarding who and how performs releases).
>>>>> >>
>>>>> >> This also allows us to introduce similar rules for new provider's
>>>>> acceptance for new providers for "main releases". It also allows running
>>>>> the "system tests" for the provider under control of the stakeholder (after
>>>>> applying AIP-47 changes).
>>>>> >>
>>>>> >> Example 1: Google team can cherry-pick changes to a
>>>>> google-provider-6 branch and then we release a google 6.8.1 or 6.9.0
>>>>> provider with some of the bug-fixes and features (together with - say
>>>>> latest 8.1.0).
>>>>> >>
>>>>> >> Example 2: DataLake provider from Databricks - can get accepted if
>>>>> Databricks commits to maintaining it. We will release the provider as long
>>>>> as Databricks maintains it.
>>>>> >>
>>>>> >> Longer context:
>>>>> >>
>>>>> >> I have - in my mind so far - a longer roadmap for providers that
>>>>> will lead them to be separated from the core and I want to write an AIP
>>>>> about that soon. This AIP will detail all the steps needed - I will work
>>>>> with multiple interested parties on it and it will take some time to agree
>>>>> and complete.  But I want to start with something tangible that will solve
>>>>> quite a few problems that were raised recently and something that seems to
>>>>> be possible to be solved in the current provider release cycle (till the
>>>>> end of June) and test some of the governance approach.
>>>>> >>
>>>>> >> This proposal simply builds on our semver approach - we do not
>>>>> change it, we just start releasing some providers (those that have some
>>>>> backing from stakeholders) in more than one version - including "latest"
>>>>> and earlier, more backwards-compatible branches. Not all providers - just
>>>>> some. Not all branches - just those that the stakeholders will commit to
>>>>> maintain.
>>>>> >>
>>>>> >> We need such a commitment from stakeholders, because we - as the
>>>>> Airflow community and maintainers, want to only actively maintain the
>>>>> latest releases, where it is in the interest of the stakeholders to
>>>>> cherry-pick and test also earlier, more backwards compatible releases of
>>>>> their choice.
>>>>> >>
>>>>> >> What problems this proposal solves:
>>>>> >>
>>>>> >> * Problem 1: DEPRECATION REMOVAL
>>>>> >>
>>>>> >> we can remove deprecations faster in "main" versions of the
>>>>> providers - no need to introduce a deprecation policy - the stakeholders
>>>>> for the providers will take care about cherry-picking and maintaining more
>>>>> "backwards-compatible" versions. We are free to remove deprecations in
>>>>> major releases (in a cherry-pickable way of course).
>>>>> >>
>>>>> >> * Problem 2: PROVIDERS DIVERGENCE
>>>>> >>
>>>>> >> We avoid the problem (already happened with the composer release)
>>>>> that the stakeholders in a given provider had to release their own version
>>>>> which was not available in their community - with some cherry-picks. We
>>>>> want to avoid "diverging" there - by releasing the cherry-picked providers
>>>>> by the community, we also give other users an opportunity to follow
>>>>> "slower" deprecation policies for as long as it is maintained.
>>>>> >>
>>>>> >> * Problem 3: PROVIDERS GOVERNANCE MODEL
>>>>> >>
>>>>> >> We are going to test a governance model that we might apply when we
>>>>> split providers. We are talking about it for quite some time - but this is
>>>>> what helps us to test the model where stakeholders provide more
>>>>> "maintenance" while the community still takes care about releases. We (as
>>>>> community) can commit to releasing such a version of a provider as long as
>>>>> the stakeholder will actively maintain it. We can stop at any moment if we
>>>>> do not have support from the stakeholder. If it works - we can keep it as a
>>>>> long-term solution. In the future we can think of other scenarios (passing
>>>>> ownership of a provider to stakeholders who want it - providing we want it
>>>>> too) but we can decide about it when we learn from the mixed-governance
>>>>> model and see if it works.
>>>>> >>
>>>>> >> * Problem 4: ACCEPTING NEW PROVIDERS
>>>>> >>
>>>>> >> If this is an acceptable approach - we can also apply a very
>>>>> similar governance model to adding new providers and that should unblock
>>>>> some of the PRs that are waiting for our decision. Knowing that we are
>>>>> going to split and that we can expect "commitment" from a stakeholder, we
>>>>> should be able to accept new providers. This might be possible assuming
>>>>> that the stakeholder will make a similar commitment - but for new
>>>>> providers, that commitment might also have to cover reviewing and testing
>>>>> new changes. We might also decide as a community to stop releasing new
>>>>> providers there if such support is missing. This way we can set the
>>>>> expectations we have as a community for new providers - we will release
>>>>> them as long as the stakeholder will actively make sure it is maintained.
>>>>> >>
>>>>> >> * Problem 5. SPLITTING PROVIDERS FROM CORE
>>>>> >>
>>>>> >> We all know we want to split providers from core. By introducing
>>>>> mixed-governance we can test if it will work for the providers before we
>>>>> split them. It will take some time (and detailed AIP) to split, but in the
>>>>> meantime we can see if we will be able to apply the mixed-governance after
>>>>> the split. We will see if we can agree when it comes to expectations and
>>>>> find solutions before we actually split.
>>>>> >>
>>>>> >> J.
>>>>>
>>>>

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Kaxil Naik <ka...@gmail.com>]

+1 -- We have discussed this during the Airflow Summit in-person with Ash,
Rafal (and his team), Jarek and I about this for a long time, and I think
this is a good step forward.

Regards,
Kaxil

On Mon, 20 Jun 2022 at 17:26, Kamil Breguła <dz...@gmail.com> wrote:

> I think we should continue to be strictly vendor-neutral. No organization
> should be able to gain special privileges or control a project’s direction.
>
> pon., 20 cze 2022 o 18:14 Kamil Breguła <dz...@gmail.com> napisał(a):
>
>> Cherry-picking to branch v2-2* or 1.10.* can only be done by the
>> committers, because only they have write permission to the apache/airflow
>> repository. As far as I know, Github does not allow us to grant write-only
>> permissions to the selected branch.
>>
>> We can keep these branches in forks managed by stakeholders teams, but I
>> am afraid of the benefit that it will be then copied by us to our
>> repository and then released by us. If the release was prepared by an
>> external team, I think we should make it clear that it was prepared by
>> another team, including by publishing on the Pypi account of the team that
>> dealt with it.
>>
>> I think that everything Apache PMC releases should be prepared and
>> created fully within the apache / airflow repositories. If stakeholders
>> team do not have such a possibility, we should figure out that these teams
>> become part of the community, and therefore work together with the entire
>> community, not in isolation. Only then will we be able to act in accordance
>> with the Apache Way <https://www.apache.org/theapacheway/>, in
>> particular each individual person will be able to contribute to the
>> community as an individual, and not as a company or stakeholders team
>> (Community of Peers) and no person will get special privileges just on the
>> basis of their employment status (Earned Authority)
>>
>>
>>
>> pon., 20 cze 2022 o 15:54 Jarek Potiuk <ja...@potiuk.com> napisał(a):
>>
>>> > Will the people who maintain the providers' packages have the commiter
>>>
>>> Nope - similarly as we do in v2-2*  or what we did in 1.10.*
>>> cherry-picking can be done in separate branches (and in this case in
>>> forks).
>>> Then the branch can be fast-forwarded by the committer in the "airflow"
>>> repo. No problem with that.
>>>
>>> J.
>>>
>>> On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <dz...@gmail.com>
>>> wrote:
>>>
>>>> Will the people who maintain the providers' packages have the commiter
>>>> status? I guess it is necessary for people to have write access to the
>>>> repository and therefore to be able to make cherry-pick changes to the
>>>> branch.
>>>>
>>>> pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org> napisał(a):
>>>> >
>>>> > +1
>>>> > From my side the proposal handles all concerns I raised in previous
>>>> threads.
>>>> > I think mixed-governance is a step in the right direction.
>>>> >
>>>> > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com>
>>>> wrote:
>>>> >>
>>>> >> Hello everyone,
>>>> >>
>>>> >> This is a follow-up after a few discussions started about providers
>>>> that were put on hold around  the summit. I held a number of discussions
>>>> during theSummit and after, and as result I think I have a proposal that
>>>> can move forward some of the "stalled" decisions we need to make.
>>>> >>
>>>> >> TL;DR;
>>>> >>
>>>> >> My proposal is a "mixed-governance" model where "stakeholders" are
>>>> more responsible for cherry-picking and testing their providers (including
>>>> system testing) while Airflow PMC members will continue to be responsible
>>>> for releasing them.
>>>> >>
>>>> >> Why do we need that?
>>>> >>
>>>> >> Google, Amazon and possibly others teams who are interested in
>>>> maintaining more backwards compatible versions of their providers will
>>>> commit to make PRs of the cherry-picks for older release branches of their
>>>> providers. Those providers we release in parallel with the latest versions
>>>> during the normal provider cycle. We can deprecate changes more
>>>> aggressively in the "latest" release if we do that.
>>>> >>
>>>> >> Those cherry-picked PRs will be driven, tested and performed by the
>>>> stakeholder teams (Google/Amazon, Databricks, others) and will only contain
>>>> cherry-picks, while we - as PMC - will release them following the ASF rules
>>>> (this is very important for the ASF to follow strict release policies
>>>> regarding who and how performs releases).
>>>> >>
>>>> >> This also allows us to introduce similar rules for new provider's
>>>> acceptance for new providers for "main releases". It also allows running
>>>> the "system tests" for the provider under control of the stakeholder (after
>>>> applying AIP-47 changes).
>>>> >>
>>>> >> Example 1: Google team can cherry-pick changes to a
>>>> google-provider-6 branch and then we release a google 6.8.1 or 6.9.0
>>>> provider with some of the bug-fixes and features (together with - say
>>>> latest 8.1.0).
>>>> >>
>>>> >> Example 2: DataLake provider from Databricks - can get accepted if
>>>> Databricks commits to maintaining it. We will release the provider as long
>>>> as Databricks maintains it.
>>>> >>
>>>> >> Longer context:
>>>> >>
>>>> >> I have - in my mind so far - a longer roadmap for providers that
>>>> will lead them to be separated from the core and I want to write an AIP
>>>> about that soon. This AIP will detail all the steps needed - I will work
>>>> with multiple interested parties on it and it will take some time to agree
>>>> and complete.  But I want to start with something tangible that will solve
>>>> quite a few problems that were raised recently and something that seems to
>>>> be possible to be solved in the current provider release cycle (till the
>>>> end of June) and test some of the governance approach.
>>>> >>
>>>> >> This proposal simply builds on our semver approach - we do not
>>>> change it, we just start releasing some providers (those that have some
>>>> backing from stakeholders) in more than one version - including "latest"
>>>> and earlier, more backwards-compatible branches. Not all providers - just
>>>> some. Not all branches - just those that the stakeholders will commit to
>>>> maintain.
>>>> >>
>>>> >> We need such a commitment from stakeholders, because we - as the
>>>> Airflow community and maintainers, want to only actively maintain the
>>>> latest releases, where it is in the interest of the stakeholders to
>>>> cherry-pick and test also earlier, more backwards compatible releases of
>>>> their choice.
>>>> >>
>>>> >> What problems this proposal solves:
>>>> >>
>>>> >> * Problem 1: DEPRECATION REMOVAL
>>>> >>
>>>> >> we can remove deprecations faster in "main" versions of the
>>>> providers - no need to introduce a deprecation policy - the stakeholders
>>>> for the providers will take care about cherry-picking and maintaining more
>>>> "backwards-compatible" versions. We are free to remove deprecations in
>>>> major releases (in a cherry-pickable way of course).
>>>> >>
>>>> >> * Problem 2: PROVIDERS DIVERGENCE
>>>> >>
>>>> >> We avoid the problem (already happened with the composer release)
>>>> that the stakeholders in a given provider had to release their own version
>>>> which was not available in their community - with some cherry-picks. We
>>>> want to avoid "diverging" there - by releasing the cherry-picked providers
>>>> by the community, we also give other users an opportunity to follow
>>>> "slower" deprecation policies for as long as it is maintained.
>>>> >>
>>>> >> * Problem 3: PROVIDERS GOVERNANCE MODEL
>>>> >>
>>>> >> We are going to test a governance model that we might apply when we
>>>> split providers. We are talking about it for quite some time - but this is
>>>> what helps us to test the model where stakeholders provide more
>>>> "maintenance" while the community still takes care about releases. We (as
>>>> community) can commit to releasing such a version of a provider as long as
>>>> the stakeholder will actively maintain it. We can stop at any moment if we
>>>> do not have support from the stakeholder. If it works - we can keep it as a
>>>> long-term solution. In the future we can think of other scenarios (passing
>>>> ownership of a provider to stakeholders who want it - providing we want it
>>>> too) but we can decide about it when we learn from the mixed-governance
>>>> model and see if it works.
>>>> >>
>>>> >> * Problem 4: ACCEPTING NEW PROVIDERS
>>>> >>
>>>> >> If this is an acceptable approach - we can also apply a very similar
>>>> governance model to adding new providers and that should unblock some of
>>>> the PRs that are waiting for our decision. Knowing that we are going to
>>>> split and that we can expect "commitment" from a stakeholder, we should be
>>>> able to accept new providers. This might be possible assuming that the
>>>> stakeholder will make a similar commitment - but for new providers, that
>>>> commitment might also have to cover reviewing and testing new changes. We
>>>> might also decide as a community to stop releasing new providers there if
>>>> such support is missing. This way we can set the expectations we have as a
>>>> community for new providers - we will release them as long as the
>>>> stakeholder will actively make sure it is maintained.
>>>> >>
>>>> >> * Problem 5. SPLITTING PROVIDERS FROM CORE
>>>> >>
>>>> >> We all know we want to split providers from core. By introducing
>>>> mixed-governance we can test if it will work for the providers before we
>>>> split them. It will take some time (and detailed AIP) to split, but in the
>>>> meantime we can see if we will be able to apply the mixed-governance after
>>>> the split. We will see if we can agree when it comes to expectations and
>>>> find solutions before we actually split.
>>>> >>
>>>> >> J.
>>>>
>>>

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Kamil Breguła <dz...@gmail.com>]

I think we should continue to be strictly vendor-neutral. No organization
should be able to gain special privileges or control a project’s direction.

pon., 20 cze 2022 o 18:14 Kamil Breguła <dz...@gmail.com> napisał(a):

> Cherry-picking to branch v2-2* or 1.10.* can only be done by the
> committers, because only they have write permission to the apache/airflow
> repository. As far as I know, Github does not allow us to grant write-only
> permissions to the selected branch.
>
> We can keep these branches in forks managed by stakeholders teams, but I
> am afraid of the benefit that it will be then copied by us to our
> repository and then released by us. If the release was prepared by an
> external team, I think we should make it clear that it was prepared by
> another team, including by publishing on the Pypi account of the team that
> dealt with it.
>
> I think that everything Apache PMC releases should be prepared and created
> fully within the apache / airflow repositories. If stakeholders team do not
> have such a possibility, we should figure out that these teams become part
> of the community, and therefore work together with the entire community,
> not in isolation. Only then will we be able to act in accordance with the
> Apache Way <https://www.apache.org/theapacheway/>, in particular each
> individual person will be able to contribute to the community as an
> individual, and not as a company or stakeholders team (Community of Peers)
> and no person will get special privileges just on the basis of their
> employment status (Earned Authority)
>
>
>
> pon., 20 cze 2022 o 15:54 Jarek Potiuk <ja...@potiuk.com> napisał(a):
>
>> > Will the people who maintain the providers' packages have the commiter
>>
>> Nope - similarly as we do in v2-2*  or what we did in 1.10.*
>> cherry-picking can be done in separate branches (and in this case in
>> forks).
>> Then the branch can be fast-forwarded by the committer in the "airflow"
>> repo. No problem with that.
>>
>> J.
>>
>> On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <dz...@gmail.com> wrote:
>>
>>> Will the people who maintain the providers' packages have the commiter
>>> status? I guess it is necessary for people to have write access to the
>>> repository and therefore to be able to make cherry-pick changes to the
>>> branch.
>>>
>>> pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org> napisał(a):
>>> >
>>> > +1
>>> > From my side the proposal handles all concerns I raised in previous
>>> threads.
>>> > I think mixed-governance is a step in the right direction.
>>> >
>>> > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com> wrote:
>>> >>
>>> >> Hello everyone,
>>> >>
>>> >> This is a follow-up after a few discussions started about providers
>>> that were put on hold around  the summit. I held a number of discussions
>>> during theSummit and after, and as result I think I have a proposal that
>>> can move forward some of the "stalled" decisions we need to make.
>>> >>
>>> >> TL;DR;
>>> >>
>>> >> My proposal is a "mixed-governance" model where "stakeholders" are
>>> more responsible for cherry-picking and testing their providers (including
>>> system testing) while Airflow PMC members will continue to be responsible
>>> for releasing them.
>>> >>
>>> >> Why do we need that?
>>> >>
>>> >> Google, Amazon and possibly others teams who are interested in
>>> maintaining more backwards compatible versions of their providers will
>>> commit to make PRs of the cherry-picks for older release branches of their
>>> providers. Those providers we release in parallel with the latest versions
>>> during the normal provider cycle. We can deprecate changes more
>>> aggressively in the "latest" release if we do that.
>>> >>
>>> >> Those cherry-picked PRs will be driven, tested and performed by the
>>> stakeholder teams (Google/Amazon, Databricks, others) and will only contain
>>> cherry-picks, while we - as PMC - will release them following the ASF rules
>>> (this is very important for the ASF to follow strict release policies
>>> regarding who and how performs releases).
>>> >>
>>> >> This also allows us to introduce similar rules for new provider's
>>> acceptance for new providers for "main releases". It also allows running
>>> the "system tests" for the provider under control of the stakeholder (after
>>> applying AIP-47 changes).
>>> >>
>>> >> Example 1: Google team can cherry-pick changes to a google-provider-6
>>> branch and then we release a google 6.8.1 or 6.9.0 provider with some of
>>> the bug-fixes and features (together with - say latest 8.1.0).
>>> >>
>>> >> Example 2: DataLake provider from Databricks - can get accepted if
>>> Databricks commits to maintaining it. We will release the provider as long
>>> as Databricks maintains it.
>>> >>
>>> >> Longer context:
>>> >>
>>> >> I have - in my mind so far - a longer roadmap for providers that will
>>> lead them to be separated from the core and I want to write an AIP about
>>> that soon. This AIP will detail all the steps needed - I will work with
>>> multiple interested parties on it and it will take some time to agree and
>>> complete.  But I want to start with something tangible that will solve
>>> quite a few problems that were raised recently and something that seems to
>>> be possible to be solved in the current provider release cycle (till the
>>> end of June) and test some of the governance approach.
>>> >>
>>> >> This proposal simply builds on our semver approach - we do not change
>>> it, we just start releasing some providers (those that have some backing
>>> from stakeholders) in more than one version - including "latest" and
>>> earlier, more backwards-compatible branches. Not all providers - just some.
>>> Not all branches - just those that the stakeholders will commit to maintain.
>>> >>
>>> >> We need such a commitment from stakeholders, because we - as the
>>> Airflow community and maintainers, want to only actively maintain the
>>> latest releases, where it is in the interest of the stakeholders to
>>> cherry-pick and test also earlier, more backwards compatible releases of
>>> their choice.
>>> >>
>>> >> What problems this proposal solves:
>>> >>
>>> >> * Problem 1: DEPRECATION REMOVAL
>>> >>
>>> >> we can remove deprecations faster in "main" versions of the providers
>>> - no need to introduce a deprecation policy - the stakeholders for the
>>> providers will take care about cherry-picking and maintaining more
>>> "backwards-compatible" versions. We are free to remove deprecations in
>>> major releases (in a cherry-pickable way of course).
>>> >>
>>> >> * Problem 2: PROVIDERS DIVERGENCE
>>> >>
>>> >> We avoid the problem (already happened with the composer release)
>>> that the stakeholders in a given provider had to release their own version
>>> which was not available in their community - with some cherry-picks. We
>>> want to avoid "diverging" there - by releasing the cherry-picked providers
>>> by the community, we also give other users an opportunity to follow
>>> "slower" deprecation policies for as long as it is maintained.
>>> >>
>>> >> * Problem 3: PROVIDERS GOVERNANCE MODEL
>>> >>
>>> >> We are going to test a governance model that we might apply when we
>>> split providers. We are talking about it for quite some time - but this is
>>> what helps us to test the model where stakeholders provide more
>>> "maintenance" while the community still takes care about releases. We (as
>>> community) can commit to releasing such a version of a provider as long as
>>> the stakeholder will actively maintain it. We can stop at any moment if we
>>> do not have support from the stakeholder. If it works - we can keep it as a
>>> long-term solution. In the future we can think of other scenarios (passing
>>> ownership of a provider to stakeholders who want it - providing we want it
>>> too) but we can decide about it when we learn from the mixed-governance
>>> model and see if it works.
>>> >>
>>> >> * Problem 4: ACCEPTING NEW PROVIDERS
>>> >>
>>> >> If this is an acceptable approach - we can also apply a very similar
>>> governance model to adding new providers and that should unblock some of
>>> the PRs that are waiting for our decision. Knowing that we are going to
>>> split and that we can expect "commitment" from a stakeholder, we should be
>>> able to accept new providers. This might be possible assuming that the
>>> stakeholder will make a similar commitment - but for new providers, that
>>> commitment might also have to cover reviewing and testing new changes. We
>>> might also decide as a community to stop releasing new providers there if
>>> such support is missing. This way we can set the expectations we have as a
>>> community for new providers - we will release them as long as the
>>> stakeholder will actively make sure it is maintained.
>>> >>
>>> >> * Problem 5. SPLITTING PROVIDERS FROM CORE
>>> >>
>>> >> We all know we want to split providers from core. By introducing
>>> mixed-governance we can test if it will work for the providers before we
>>> split them. It will take some time (and detailed AIP) to split, but in the
>>> meantime we can see if we will be able to apply the mixed-governance after
>>> the split. We will see if we can agree when it comes to expectations and
>>> find solutions before we actually split.
>>> >>
>>> >> J.
>>>
>>

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Kamil Breguła <dz...@gmail.com>]

Cherry-picking to branch v2-2* or 1.10.* can only be done by the
committers, because only they have write permission to the apache/airflow
repository. As far as I know, Github does not allow us to grant write-only
permissions to the selected branch.

We can keep these branches in forks managed by stakeholders teams, but I am
afraid of the benefit that it will be then copied by us to our repository
and then released by us. If the release was prepared by an external team, I
think we should make it clear that it was prepared by another team,
including by publishing on the Pypi account of the team that dealt with it.

I think that everything Apache PMC releases should be prepared and created
fully within the apache / airflow repositories. If stakeholders team do not
have such a possibility, we should figure out that these teams become part
of the community, and therefore work together with the entire community,
not in isolation. Only then will we be able to act in accordance with the
Apache Way <https://www.apache.org/theapacheway/>, in particular each
individual person will be able to contribute to the community as an
individual, and not as a company or stakeholders team (Community of Peers)
and no person will get special privileges just on the basis of their
employment status (Earned Authority)



pon., 20 cze 2022 o 15:54 Jarek Potiuk <ja...@potiuk.com> napisał(a):

> > Will the people who maintain the providers' packages have the commiter
>
> Nope - similarly as we do in v2-2*  or what we did in 1.10.*
> cherry-picking can be done in separate branches (and in this case in
> forks).
> Then the branch can be fast-forwarded by the committer in the "airflow"
> repo. No problem with that.
>
> J.
>
> On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <dz...@gmail.com> wrote:
>
>> Will the people who maintain the providers' packages have the commiter
>> status? I guess it is necessary for people to have write access to the
>> repository and therefore to be able to make cherry-pick changes to the
>> branch.
>>
>> pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org> napisał(a):
>> >
>> > +1
>> > From my side the proposal handles all concerns I raised in previous
>> threads.
>> > I think mixed-governance is a step in the right direction.
>> >
>> > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com> wrote:
>> >>
>> >> Hello everyone,
>> >>
>> >> This is a follow-up after a few discussions started about providers
>> that were put on hold around  the summit. I held a number of discussions
>> during theSummit and after, and as result I think I have a proposal that
>> can move forward some of the "stalled" decisions we need to make.
>> >>
>> >> TL;DR;
>> >>
>> >> My proposal is a "mixed-governance" model where "stakeholders" are
>> more responsible for cherry-picking and testing their providers (including
>> system testing) while Airflow PMC members will continue to be responsible
>> for releasing them.
>> >>
>> >> Why do we need that?
>> >>
>> >> Google, Amazon and possibly others teams who are interested in
>> maintaining more backwards compatible versions of their providers will
>> commit to make PRs of the cherry-picks for older release branches of their
>> providers. Those providers we release in parallel with the latest versions
>> during the normal provider cycle. We can deprecate changes more
>> aggressively in the "latest" release if we do that.
>> >>
>> >> Those cherry-picked PRs will be driven, tested and performed by the
>> stakeholder teams (Google/Amazon, Databricks, others) and will only contain
>> cherry-picks, while we - as PMC - will release them following the ASF rules
>> (this is very important for the ASF to follow strict release policies
>> regarding who and how performs releases).
>> >>
>> >> This also allows us to introduce similar rules for new provider's
>> acceptance for new providers for "main releases". It also allows running
>> the "system tests" for the provider under control of the stakeholder (after
>> applying AIP-47 changes).
>> >>
>> >> Example 1: Google team can cherry-pick changes to a google-provider-6
>> branch and then we release a google 6.8.1 or 6.9.0 provider with some of
>> the bug-fixes and features (together with - say latest 8.1.0).
>> >>
>> >> Example 2: DataLake provider from Databricks - can get accepted if
>> Databricks commits to maintaining it. We will release the provider as long
>> as Databricks maintains it.
>> >>
>> >> Longer context:
>> >>
>> >> I have - in my mind so far - a longer roadmap for providers that will
>> lead them to be separated from the core and I want to write an AIP about
>> that soon. This AIP will detail all the steps needed - I will work with
>> multiple interested parties on it and it will take some time to agree and
>> complete.  But I want to start with something tangible that will solve
>> quite a few problems that were raised recently and something that seems to
>> be possible to be solved in the current provider release cycle (till the
>> end of June) and test some of the governance approach.
>> >>
>> >> This proposal simply builds on our semver approach - we do not change
>> it, we just start releasing some providers (those that have some backing
>> from stakeholders) in more than one version - including "latest" and
>> earlier, more backwards-compatible branches. Not all providers - just some.
>> Not all branches - just those that the stakeholders will commit to maintain.
>> >>
>> >> We need such a commitment from stakeholders, because we - as the
>> Airflow community and maintainers, want to only actively maintain the
>> latest releases, where it is in the interest of the stakeholders to
>> cherry-pick and test also earlier, more backwards compatible releases of
>> their choice.
>> >>
>> >> What problems this proposal solves:
>> >>
>> >> * Problem 1: DEPRECATION REMOVAL
>> >>
>> >> we can remove deprecations faster in "main" versions of the providers
>> - no need to introduce a deprecation policy - the stakeholders for the
>> providers will take care about cherry-picking and maintaining more
>> "backwards-compatible" versions. We are free to remove deprecations in
>> major releases (in a cherry-pickable way of course).
>> >>
>> >> * Problem 2: PROVIDERS DIVERGENCE
>> >>
>> >> We avoid the problem (already happened with the composer release) that
>> the stakeholders in a given provider had to release their own version which
>> was not available in their community - with some cherry-picks. We want to
>> avoid "diverging" there - by releasing the cherry-picked providers by the
>> community, we also give other users an opportunity to follow "slower"
>> deprecation policies for as long as it is maintained.
>> >>
>> >> * Problem 3: PROVIDERS GOVERNANCE MODEL
>> >>
>> >> We are going to test a governance model that we might apply when we
>> split providers. We are talking about it for quite some time - but this is
>> what helps us to test the model where stakeholders provide more
>> "maintenance" while the community still takes care about releases. We (as
>> community) can commit to releasing such a version of a provider as long as
>> the stakeholder will actively maintain it. We can stop at any moment if we
>> do not have support from the stakeholder. If it works - we can keep it as a
>> long-term solution. In the future we can think of other scenarios (passing
>> ownership of a provider to stakeholders who want it - providing we want it
>> too) but we can decide about it when we learn from the mixed-governance
>> model and see if it works.
>> >>
>> >> * Problem 4: ACCEPTING NEW PROVIDERS
>> >>
>> >> If this is an acceptable approach - we can also apply a very similar
>> governance model to adding new providers and that should unblock some of
>> the PRs that are waiting for our decision. Knowing that we are going to
>> split and that we can expect "commitment" from a stakeholder, we should be
>> able to accept new providers. This might be possible assuming that the
>> stakeholder will make a similar commitment - but for new providers, that
>> commitment might also have to cover reviewing and testing new changes. We
>> might also decide as a community to stop releasing new providers there if
>> such support is missing. This way we can set the expectations we have as a
>> community for new providers - we will release them as long as the
>> stakeholder will actively make sure it is maintained.
>> >>
>> >> * Problem 5. SPLITTING PROVIDERS FROM CORE
>> >>
>> >> We all know we want to split providers from core. By introducing
>> mixed-governance we can test if it will work for the providers before we
>> split them. It will take some time (and detailed AIP) to split, but in the
>> meantime we can see if we will be able to apply the mixed-governance after
>> the split. We will see if we can agree when it comes to expectations and
>> find solutions before we actually split.
>> >>
>> >> J.
>>
>

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Jarek Potiuk <ja...@potiuk.com>]

> Will the people who maintain the providers' packages have the commiter

Nope - similarly as we do in v2-2*  or what we did in 1.10.*
cherry-picking can be done in separate branches (and in this case in
forks).
Then the branch can be fast-forwarded by the committer in the "airflow"
repo. No problem with that.

J.

On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <dz...@gmail.com> wrote:

> Will the people who maintain the providers' packages have the commiter
> status? I guess it is necessary for people to have write access to the
> repository and therefore to be able to make cherry-pick changes to the
> branch.
>
> pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org> napisał(a):
> >
> > +1
> > From my side the proposal handles all concerns I raised in previous
> threads.
> > I think mixed-governance is a step in the right direction.
> >
> > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com> wrote:
> >>
> >> Hello everyone,
> >>
> >> This is a follow-up after a few discussions started about providers
> that were put on hold around  the summit. I held a number of discussions
> during theSummit and after, and as result I think I have a proposal that
> can move forward some of the "stalled" decisions we need to make.
> >>
> >> TL;DR;
> >>
> >> My proposal is a "mixed-governance" model where "stakeholders" are more
> responsible for cherry-picking and testing their providers (including
> system testing) while Airflow PMC members will continue to be responsible
> for releasing them.
> >>
> >> Why do we need that?
> >>
> >> Google, Amazon and possibly others teams who are interested in
> maintaining more backwards compatible versions of their providers will
> commit to make PRs of the cherry-picks for older release branches of their
> providers. Those providers we release in parallel with the latest versions
> during the normal provider cycle. We can deprecate changes more
> aggressively in the "latest" release if we do that.
> >>
> >> Those cherry-picked PRs will be driven, tested and performed by the
> stakeholder teams (Google/Amazon, Databricks, others) and will only contain
> cherry-picks, while we - as PMC - will release them following the ASF rules
> (this is very important for the ASF to follow strict release policies
> regarding who and how performs releases).
> >>
> >> This also allows us to introduce similar rules for new provider's
> acceptance for new providers for "main releases". It also allows running
> the "system tests" for the provider under control of the stakeholder (after
> applying AIP-47 changes).
> >>
> >> Example 1: Google team can cherry-pick changes to a google-provider-6
> branch and then we release a google 6.8.1 or 6.9.0 provider with some of
> the bug-fixes and features (together with - say latest 8.1.0).
> >>
> >> Example 2: DataLake provider from Databricks - can get accepted if
> Databricks commits to maintaining it. We will release the provider as long
> as Databricks maintains it.
> >>
> >> Longer context:
> >>
> >> I have - in my mind so far - a longer roadmap for providers that will
> lead them to be separated from the core and I want to write an AIP about
> that soon. This AIP will detail all the steps needed - I will work with
> multiple interested parties on it and it will take some time to agree and
> complete.  But I want to start with something tangible that will solve
> quite a few problems that were raised recently and something that seems to
> be possible to be solved in the current provider release cycle (till the
> end of June) and test some of the governance approach.
> >>
> >> This proposal simply builds on our semver approach - we do not change
> it, we just start releasing some providers (those that have some backing
> from stakeholders) in more than one version - including "latest" and
> earlier, more backwards-compatible branches. Not all providers - just some.
> Not all branches - just those that the stakeholders will commit to maintain.
> >>
> >> We need such a commitment from stakeholders, because we - as the
> Airflow community and maintainers, want to only actively maintain the
> latest releases, where it is in the interest of the stakeholders to
> cherry-pick and test also earlier, more backwards compatible releases of
> their choice.
> >>
> >> What problems this proposal solves:
> >>
> >> * Problem 1: DEPRECATION REMOVAL
> >>
> >> we can remove deprecations faster in "main" versions of the providers -
> no need to introduce a deprecation policy - the stakeholders for the
> providers will take care about cherry-picking and maintaining more
> "backwards-compatible" versions. We are free to remove deprecations in
> major releases (in a cherry-pickable way of course).
> >>
> >> * Problem 2: PROVIDERS DIVERGENCE
> >>
> >> We avoid the problem (already happened with the composer release) that
> the stakeholders in a given provider had to release their own version which
> was not available in their community - with some cherry-picks. We want to
> avoid "diverging" there - by releasing the cherry-picked providers by the
> community, we also give other users an opportunity to follow "slower"
> deprecation policies for as long as it is maintained.
> >>
> >> * Problem 3: PROVIDERS GOVERNANCE MODEL
> >>
> >> We are going to test a governance model that we might apply when we
> split providers. We are talking about it for quite some time - but this is
> what helps us to test the model where stakeholders provide more
> "maintenance" while the community still takes care about releases. We (as
> community) can commit to releasing such a version of a provider as long as
> the stakeholder will actively maintain it. We can stop at any moment if we
> do not have support from the stakeholder. If it works - we can keep it as a
> long-term solution. In the future we can think of other scenarios (passing
> ownership of a provider to stakeholders who want it - providing we want it
> too) but we can decide about it when we learn from the mixed-governance
> model and see if it works.
> >>
> >> * Problem 4: ACCEPTING NEW PROVIDERS
> >>
> >> If this is an acceptable approach - we can also apply a very similar
> governance model to adding new providers and that should unblock some of
> the PRs that are waiting for our decision. Knowing that we are going to
> split and that we can expect "commitment" from a stakeholder, we should be
> able to accept new providers. This might be possible assuming that the
> stakeholder will make a similar commitment - but for new providers, that
> commitment might also have to cover reviewing and testing new changes. We
> might also decide as a community to stop releasing new providers there if
> such support is missing. This way we can set the expectations we have as a
> community for new providers - we will release them as long as the
> stakeholder will actively make sure it is maintained.
> >>
> >> * Problem 5. SPLITTING PROVIDERS FROM CORE
> >>
> >> We all know we want to split providers from core. By introducing
> mixed-governance we can test if it will work for the providers before we
> split them. It will take some time (and detailed AIP) to split, but in the
> meantime we can see if we will be able to apply the mixed-governance after
> the split. We will see if we can agree when it comes to expectations and
> find solutions before we actually split.
> >>
> >> J.
>

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Kamil Breguła <dz...@gmail.com>]

Will the people who maintain the providers' packages have the commiter
status? I guess it is necessary for people to have write access to the
repository and therefore to be able to make cherry-pick changes to the
branch.

pon., 20 cze 2022 o 09:13 Elad Kalif <el...@apache.org> napisał(a):
>
> +1
> From my side the proposal handles all concerns I raised in previous threads.
> I think mixed-governance is a step in the right direction.
>
> On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com> wrote:
>>
>> Hello everyone,
>>
>> This is a follow-up after a few discussions started about providers that were put on hold around  the summit. I held a number of discussions during theSummit and after, and as result I think I have a proposal that can move forward some of the "stalled" decisions we need to make.
>>
>> TL;DR;
>>
>> My proposal is a "mixed-governance" model where "stakeholders" are more responsible for cherry-picking and testing their providers (including system testing) while Airflow PMC members will continue to be responsible for releasing them.
>>
>> Why do we need that?
>>
>> Google, Amazon and possibly others teams who are interested in maintaining more backwards compatible versions of their providers will commit to make PRs of the cherry-picks for older release branches of their providers. Those providers we release in parallel with the latest versions during the normal provider cycle. We can deprecate changes more aggressively in the "latest" release if we do that.
>>
>> Those cherry-picked PRs will be driven, tested and performed by the stakeholder teams (Google/Amazon, Databricks, others) and will only contain cherry-picks, while we - as PMC - will release them following the ASF rules (this is very important for the ASF to follow strict release policies regarding who and how performs releases).
>>
>> This also allows us to introduce similar rules for new provider's acceptance for new providers for "main releases". It also allows running the "system tests" for the provider under control of the stakeholder (after applying AIP-47 changes).
>>
>> Example 1: Google team can cherry-pick changes to a google-provider-6 branch and then we release a google 6.8.1 or 6.9.0 provider with some of the bug-fixes and features (together with - say latest 8.1.0).
>>
>> Example 2: DataLake provider from Databricks - can get accepted if Databricks commits to maintaining it. We will release the provider as long as Databricks maintains it.
>>
>> Longer context:
>>
>> I have - in my mind so far - a longer roadmap for providers that will lead them to be separated from the core and I want to write an AIP about that soon. This AIP will detail all the steps needed - I will work with multiple interested parties on it and it will take some time to agree and complete.  But I want to start with something tangible that will solve quite a few problems that were raised recently and something that seems to be possible to be solved in the current provider release cycle (till the end of June) and test some of the governance approach.
>>
>> This proposal simply builds on our semver approach - we do not change it, we just start releasing some providers (those that have some backing from stakeholders) in more than one version - including "latest" and earlier, more backwards-compatible branches. Not all providers - just some. Not all branches - just those that the stakeholders will commit to maintain.
>>
>> We need such a commitment from stakeholders, because we - as the  Airflow community and maintainers, want to only actively maintain the latest releases, where it is in the interest of the stakeholders to cherry-pick and test also earlier, more backwards compatible releases of their choice.
>>
>> What problems this proposal solves:
>>
>> * Problem 1: DEPRECATION REMOVAL
>>
>> we can remove deprecations faster in "main" versions of the providers - no need to introduce a deprecation policy - the stakeholders for the providers will take care about cherry-picking and maintaining more "backwards-compatible" versions. We are free to remove deprecations in major releases (in a cherry-pickable way of course).
>>
>> * Problem 2: PROVIDERS DIVERGENCE
>>
>> We avoid the problem (already happened with the composer release) that the stakeholders in a given provider had to release their own version which was not available in their community - with some cherry-picks. We want to avoid "diverging" there - by releasing the cherry-picked providers by the community, we also give other users an opportunity to follow "slower" deprecation policies for as long as it is maintained.
>>
>> * Problem 3: PROVIDERS GOVERNANCE MODEL
>>
>> We are going to test a governance model that we might apply when we split providers. We are talking about it for quite some time - but this is what helps us to test the model where stakeholders provide more "maintenance" while the community still takes care about releases. We (as community) can commit to releasing such a version of a provider as long as the stakeholder will actively maintain it. We can stop at any moment if we do not have support from the stakeholder. If it works - we can keep it as a long-term solution. In the future we can think of other scenarios (passing ownership of a provider to stakeholders who want it - providing we want it too) but we can decide about it when we learn from the mixed-governance model and see if it works.
>>
>> * Problem 4: ACCEPTING NEW PROVIDERS
>>
>> If this is an acceptable approach - we can also apply a very similar governance model to adding new providers and that should unblock some of the PRs that are waiting for our decision. Knowing that we are going to split and that we can expect "commitment" from a stakeholder, we should be able to accept new providers. This might be possible assuming that the stakeholder will make a similar commitment - but for new providers, that commitment might also have to cover reviewing and testing new changes. We might also decide as a community to stop releasing new providers there if such support is missing. This way we can set the expectations we have as a community for new providers - we will release them as long as the stakeholder will actively make sure it is maintained.
>>
>> * Problem 5. SPLITTING PROVIDERS FROM CORE
>>
>> We all know we want to split providers from core. By introducing mixed-governance we can test if it will work for the providers before we split them. It will take some time (and detailed AIP) to split, but in the meantime we can see if we will be able to apply the mixed-governance after the split. We will see if we can agree when it comes to expectations and find solutions before we actually split.
>>
>> J.

Re: [PROPOSAL] Provider's mixed governance model - first step of provider separation

[Elad Kalif <el...@apache.org>]

+1
From my side the proposal handles all concerns I raised in previous threads.
I think mixed-governance is a step in the right direction.

On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <ja...@potiuk.com> wrote:

> Hello everyone,
>
> This is a follow-up after a few discussions started about providers that
> were put on hold around  the summit. I held a number of discussions during
> theSummit and after, and as result I think I have a proposal that can move
> forward some of the "stalled" decisions we need to make.
>
> *TL;DR;*
>
> My proposal is a "mixed-governance" model where "stakeholders" are more
> responsible for cherry-picking and testing their providers (including
> system testing) while Airflow PMC members will continue to be responsible
> for releasing them.
>
> *Why do we need that?*
>
> Google, Amazon and possibly others teams who are interested in maintaining
> more backwards compatible versions of their providers will commit to make
> PRs of the cherry-picks for older release branches of their providers.
> Those providers we release in parallel with the latest versions during the
> normal provider cycle. We can deprecate changes more aggressively in the
> "latest" release if we do that.
>
> Those cherry-picked PRs will be driven, tested and performed by the
> stakeholder teams (Google/Amazon, Databricks, others) and will only contain
> cherry-picks, while we - as PMC - will release them following the ASF rules
> (this is very important for the ASF to follow strict release policies
> regarding who and how performs releases).
>
> This also allows us to introduce similar rules for new provider's
> acceptance for new providers for "main releases". It also allows running
> the "system tests" for the provider under control of the stakeholder (after
> applying AIP-47 changes).
>
> *Example 1*: Google team can cherry-pick changes to a google-provider-6
> branch and then we release a google 6.8.1 or 6.9.0 provider with some of
> the bug-fixes and features (together with - say latest 8.1.0).
>
> *Example 2*: DataLake provider from Databricks - can get accepted if
> Databricks commits to maintaining it. We will release the provider as long
> as Databricks maintains it.
>
> *Longer context:*
>
> I have - in my mind so far - a longer roadmap for providers that will lead
> them to be separated from the core and I want to write an AIP about that
> soon. This AIP will detail all the steps needed - I will work with multiple
> interested parties on it and it will take some time to agree and complete.
> But I want to start with something tangible that will solve quite a few
> problems that were raised recently and something that seems to be possible
> to be solved in the current provider release cycle (till the end of June)
> and test some of the governance approach.
>
> This proposal simply builds on our semver approach - we do not change it,
> we just start releasing some providers (those that have some backing from
> stakeholders) in more than one version - including "latest" and earlier,
> more backwards-compatible branches. Not all providers - just some. Not all
> branches - just those that the stakeholders will commit to maintain.
>
> We need such a commitment from stakeholders, because we - as the
> Airflow community and maintainers, want to only actively maintain the
> latest releases, where it is in the interest of the stakeholders to
> cherry-pick and test also earlier, more backwards compatible releases of
> their choice.
>
> *What problems this proposal solves:*
>
> ** Problem 1*: DEPRECATION REMOVAL
>
> we can remove deprecations faster in "main" versions of the providers - no
> need to introduce a deprecation policy - the stakeholders for the providers
> will take care about cherry-picking and maintaining more
> "backwards-compatible" versions. We are free to remove deprecations in
> major releases (in a cherry-pickable way of course).
>
> ** Problem 2*: PROVIDERS DIVERGENCE
>
> We avoid the problem (already happened with the composer release) that the
> stakeholders in a given provider had to release their own version which was
> not available in their community - with some cherry-picks. We want to avoid
> "diverging" there - by releasing the cherry-picked providers by the
> community, we also give other users an opportunity to follow "slower"
> deprecation policies for as long as it is maintained.
>
> ** Problem 3*: PROVIDERS GOVERNANCE MODEL
>
> We are going to test a governance model that we might apply when we split
> providers. We are talking about it for quite some time - but this is what
> helps us to test the model where stakeholders provide more "maintenance"
> while the community still takes care about releases. We (as community) can
> commit to releasing such a version of a provider as long as the stakeholder
> will actively maintain it. We can stop at any moment if we do not have
> support from the stakeholder. If it works - we can keep it as a long-term
> solution. In the future we can think of other scenarios (passing ownership
> of a provider to stakeholders who want it - providing we want it too) but
> we can decide about it when we learn from the mixed-governance model and
> see if it works.
>
> ** Problem 4*: ACCEPTING NEW PROVIDERS
>
> If this is an acceptable approach - we can also apply a very
> similar governance model to adding new providers and that should unblock
> some of the PRs that are waiting for our decision. Knowing that we are
> going to split and that we can expect "commitment" from a stakeholder, we
> should be able to accept new providers. This might be possible assuming
> that the stakeholder will make a similar commitment - but for new
> providers, that commitment might also have to cover reviewing and testing
> new changes. We might also decide as a community to stop releasing new
> providers there if such support is missing. This way we can set the
> expectations we have as a community for new providers - we will release
> them as long as the stakeholder will actively make sure it is maintained.
>
> ** Problem 5.* SPLITTING PROVIDERS FROM CORE
>
> We all know we want to split providers from core. By introducing
> mixed-governance we can test if it will work for the providers before we
> split them. It will take some time (and detailed AIP) to split, but in the
> meantime we can see if we will be able to apply the mixed-governance after
> the split. We will see if we can agree when it comes to expectations and
> find solutions before we actually split.
>
> J.
>