You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Jarek Potiuk <ja...@potiuk.com> on 2025/03/21 22:09:14 UTC
[DISCUSS] Decisions made on devlist
I also would like to start another thread regarding decision making in our
projects. We had some discussions about decision making in our project.
As an ASF project we are supposed to make important decisions on the
devlist. Full stop. "If it did not happen at the devlist, it did not
happen" is something that you can hear often in the ASF - even if the only
place you can actually see it written this way is
https://www.apache.org/press/highlights.html#2017
The Apache Software Foundation has a very clear notion of important
decision making:
* official voting rules https://www.apache.org/foundation/voting.html
* community development guidelines
https://community.apache.org/committers/decisionMaking.html
Both of those refer to decision making at the devlist.
We had a few - unnecessarily heated - discussions on how decisions are made
in the project. I personally think that our decision making should be done
at the devlist - where anyone can participate. Things like versioning rules
for API (hinted by Ash at the last dev call) or how our repo is structured
(a new proposal raised today by TP) should - IMHO - be discussed here, at
the devlist.
Not in a slack thread, not in private discussion, not when two or more
people talk to each other in a call. It's fine to discuss things outside -
of course, but then any proposals for a change of things that we already
discussed for weeks and either implicitly (by non-objection) or explicitly
(by not responding to [LAZY CONSENSUS] or [VOTE] thread in the devlist) are
just discussions. If someone wants a change, starting a thread in devlist
is the right way of proposing a change. Especially for things that were
discussed before - sometimes for weeks, and no concerns were raised.
IMHO - it's very simple - want a change - start and lead a [DISCUSS]. [LAZY
CONSENSUS]. or [VOTE] (depending on the level of disagreement and number of
different opinions). Not very complex - it just requires to start and lead
a discussion mailing list thread. Super inclusive and follows the archiving
and all other requirements of the ASF (for example you do not have to agree
TOC of Slack to participate)
I would love to hear if others disagree with it and think that this process
is overly complex or problematic. I think it's quite clear, reasonable and
great to keep community decision making in check and follow all the rules
and expectations of the ASF, but maybe there are some concerns with the
process and we would like to improve it.
I would love to hear what others think about it.
J.
Re: [DISCUSS] Decisions made on devlist
Posted by Jarek Potiuk <ja...@potiuk.com>.
To be very concrete: I think my proposal is just to exercise a bit of
empathy towards other work and when you are asked "let's discuss it on a
devlist" because it's a complex case, with huge impact (and effort needed)
and potentially disruptive and where many people will like have opinion,
let's just do it. Especially if the previous incarnation of it has already
been discussed and actions taken.
To the "bike shed color argument" of Ash.
Yes, maybe choosing a color seems like bike-shed discussion, but when you
want to repaint your whole house in a different colour while there are
about a 100 people at the same time in your flat moving the furniture,
polishing the floor, changing the windows and replacing door-knobs. You
painted half of the flat and you prepared it carefully and planned around
all the other work so that the places you already painted can be safely
used by those 100s of people while you are painting the rest of the house.
You agreed the color with every stakeholder and you found a good one you
paint it with, and it took weeks to paint half of the house and the rest is
planned.
Then you have one of the tenants come and say "I do not like the green,
let's make it red altogether". Surely you do not want to jump on it, and
do it straight away or even discuss the new colour on the corridor with
that one stakeholder. And certainly you do not want to repaint it all over
again while splashing the paint on all the people that are at the same time
in the rooms that you already painted.
Likely you want to gather those stakeholders afterwards and agree on a new
colour - but possibly in a bit calmer way, after all the frenzy is done.
And maybe even start discussing on the new colour, and whether it makes
sense at all, or maybe the current green colour is really acceptable, and
only one of the tenants has a different opinion and the rest prefers green?
It would seem so from the previous discussions, let's check it on devlist.
That is my proposal.
J.
On Wed, Mar 26, 2025 at 6:07 PM Daniel Standish
<da...@astronomer.io.invalid> wrote:
> BUT LET ME ADD
>
> Can anyone sort of try to focus the discussion here a bit?
>
> Is there anything specific being proposed?
>
> On Wed, Mar 26, 2025 at 9:58 AM Daniel Standish <
> daniel.standish@astronomer.io> wrote:
>
> > I think that many times, you only bring things to list after they reach a
> > certain level of maturity. Sometimes on slack etc you are just working
> out
> > how you feel about something, trying to understand something. I think it
> > is perfectly fine to have such conversations on slack.
> >
> > Perhaps you find that there is consensus but you want to inform folks on
> > dev list. Then you can make a lazy consensus thread.
> >
> > Perhaps you find that there is disagreement and you need community input
> > to help resolve. Then you can make a discuss thread.
> >
> > Or perhaps you realize that there is more or less agreement (or
> > disagreement but acceptance) and there is no need to bring it to list.
> >
> > All such outcomes are possible.
> >
>
Re: [DISCUSS] Decisions made on devlist
Posted by Daniel Standish <da...@astronomer.io.INVALID>.
BUT LET ME ADD
Can anyone sort of try to focus the discussion here a bit?
Is there anything specific being proposed?
On Wed, Mar 26, 2025 at 9:58 AM Daniel Standish <
daniel.standish@astronomer.io> wrote:
> I think that many times, you only bring things to list after they reach a
> certain level of maturity. Sometimes on slack etc you are just working out
> how you feel about something, trying to understand something. I think it
> is perfectly fine to have such conversations on slack.
>
> Perhaps you find that there is consensus but you want to inform folks on
> dev list. Then you can make a lazy consensus thread.
>
> Perhaps you find that there is disagreement and you need community input
> to help resolve. Then you can make a discuss thread.
>
> Or perhaps you realize that there is more or less agreement (or
> disagreement but acceptance) and there is no need to bring it to list.
>
> All such outcomes are possible.
>
Re: [DISCUSS] Decisions made on devlist
Posted by Daniel Standish <da...@astronomer.io.INVALID>.
I think that many times, you only bring things to list after they reach a
certain level of maturity. Sometimes on slack etc you are just working out
how you feel about something, trying to understand something. I think it
is perfectly fine to have such conversations on slack.
Perhaps you find that there is consensus but you want to inform folks on
dev list. Then you can make a lazy consensus thread.
Perhaps you find that there is disagreement and you need community input to
help resolve. Then you can make a discuss thread.
Or perhaps you realize that there is more or less agreement (or
disagreement but acceptance) and there is no need to bring it to list.
All such outcomes are possible.
Re: [DISCUSS] Decisions made on devlist
Posted by Abhishek Bhakat <ab...@astronomer.io.INVALID>.
IMO having the discussion on Github has very helpful for me. That's just
the first place we look for.
And if been asked on why was "*this particular change*" was made, I
can answer it brought up a discussion on the dev list and the
community voted on it.
In a gist, I don't have a problem with the current process or say I
experienced a scenario where wasn't able to find why a certain decision
was taken.
- Avi
On Tue, Mar 25, 2025 at 1:21 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> > So I’m not sure what you are actually proposing, since we already do make
> important decisions on the list, and that discussing the minutia of every
> decision isn’t practical.
>
> I think it's really a question of the impact it has. In this case changing
> the layout of the repo (as I tried to explain here and in the slack thread)
> has a huge impact on everyone, takes weeks of effort to implement - you
> experienced it yourself - and require people to re-learn how to contribute
> (we have 3200+ contributors and maybe 180 of them active in any
> given week). And I think it's good to hear what's their opinion is - that's
> what the devlist community is about.
>
> All that I am asking about - let's bring important discussions about
> decisions that are affecting both our users and contributors - to the
> devlist. Not every single decision. Just the ones that affect pretty much
> everyone's job, are difficult to revert and take weeks to discuss, agree
> and implement,
>
> Layout of the repo is one of those cases.
>
> J.
>
>
>
> On Tue, Mar 25, 2025 at 1:13 PM Ash Berlin-Taylor <as...@apache.org> wrote:
>
> >
> >
> > > On 23 Mar 2025, at 03:44, Jarek Potiuk <ja...@potiuk.com> wrote:
> > >
> > > The thread here was sparked with a slack discussion where TP
> > > commented essentially "we should change our layout of folder, I do not
> > like
> > > the lack of consistency" - in a random slack conversation, that many of
> > us
> > > could have missed. And I was called out essentially for saying "We've
> > been
> > > discussing it for weeks on the devlist, please bring it to devlist and
> > > let's discuss it there”.
> >
> > This is where you and I differ. You viewed this as a real proposal or a
> > demand for change. I view it as a venting of frustration only.
> >
> > My tl;dr on this whole thread: yes of course we should discuss important
> > or meaningful topics on the mailing list. But I have a very different
> view
> > on what actually counts as significant.
> >
> > To take your argument to an extreme (that I know you aren’t proposing,
> > it's just for illustration) is that every code change needs to be
> discussed
> > on the mailing list. Now clearly that is absurd, so there has to be some
> > threshold where discussing in a GH issue/PR or Slack is fine.
> >
> > Is the name of the folder which we keep our source code in the repo
> > important? No, it’s just above a bike shed color argument at best. If
> > someone wants to discuss it on the list, go for it. In my mind, if
> someone
> > doesn’t and wants to discuss it on a PR and gets enough feedback there:
> > that is also fine to me. The PR speaks for itself, and it is not an
> > important historical decision, and arguably: finding the reasoning when
> it
> > is done in a GH pr is easier than having to search the mailing list
> > archive, both are equally as permanent.
> >
> > So I’m not sure what you are actually proposing, since we already do make
> > important decisions on the list, and that discussing the minutia of every
> > decision isn’t practical.
> >
> > -ash
> >
> >
>
Re: [DISCUSS] Decisions made on devlist
Posted by Jarek Potiuk <ja...@potiuk.com>.
> So I’m not sure what you are actually proposing, since we already do make
important decisions on the list, and that discussing the minutia of every
decision isn’t practical.
I think it's really a question of the impact it has. In this case changing
the layout of the repo (as I tried to explain here and in the slack thread)
has a huge impact on everyone, takes weeks of effort to implement - you
experienced it yourself - and require people to re-learn how to contribute
(we have 3200+ contributors and maybe 180 of them active in any
given week). And I think it's good to hear what's their opinion is - that's
what the devlist community is about.
All that I am asking about - let's bring important discussions about
decisions that are affecting both our users and contributors - to the
devlist. Not every single decision. Just the ones that affect pretty much
everyone's job, are difficult to revert and take weeks to discuss, agree
and implement,
Layout of the repo is one of those cases.
J.
On Tue, Mar 25, 2025 at 1:13 PM Ash Berlin-Taylor <as...@apache.org> wrote:
>
>
> > On 23 Mar 2025, at 03:44, Jarek Potiuk <ja...@potiuk.com> wrote:
> >
> > The thread here was sparked with a slack discussion where TP
> > commented essentially "we should change our layout of folder, I do not
> like
> > the lack of consistency" - in a random slack conversation, that many of
> us
> > could have missed. And I was called out essentially for saying "We've
> been
> > discussing it for weeks on the devlist, please bring it to devlist and
> > let's discuss it there”.
>
> This is where you and I differ. You viewed this as a real proposal or a
> demand for change. I view it as a venting of frustration only.
>
> My tl;dr on this whole thread: yes of course we should discuss important
> or meaningful topics on the mailing list. But I have a very different view
> on what actually counts as significant.
>
> To take your argument to an extreme (that I know you aren’t proposing,
> it's just for illustration) is that every code change needs to be discussed
> on the mailing list. Now clearly that is absurd, so there has to be some
> threshold where discussing in a GH issue/PR or Slack is fine.
>
> Is the name of the folder which we keep our source code in the repo
> important? No, it’s just above a bike shed color argument at best. If
> someone wants to discuss it on the list, go for it. In my mind, if someone
> doesn’t and wants to discuss it on a PR and gets enough feedback there:
> that is also fine to me. The PR speaks for itself, and it is not an
> important historical decision, and arguably: finding the reasoning when it
> is done in a GH pr is easier than having to search the mailing list
> archive, both are equally as permanent.
>
> So I’m not sure what you are actually proposing, since we already do make
> important decisions on the list, and that discussing the minutia of every
> decision isn’t practical.
>
> -ash
>
>
Re: [DISCUSS] Decisions made on devlist
Posted by Ash Berlin-Taylor <as...@apache.org>.
> On 23 Mar 2025, at 03:44, Jarek Potiuk <ja...@potiuk.com> wrote:
>
> The thread here was sparked with a slack discussion where TP
> commented essentially "we should change our layout of folder, I do not like
> the lack of consistency" - in a random slack conversation, that many of us
> could have missed. And I was called out essentially for saying "We've been
> discussing it for weeks on the devlist, please bring it to devlist and
> let's discuss it there”.
This is where you and I differ. You viewed this as a real proposal or a demand for change. I view it as a venting of frustration only.
My tl;dr on this whole thread: yes of course we should discuss important or meaningful topics on the mailing list. But I have a very different view on what actually counts as significant.
To take your argument to an extreme (that I know you aren’t proposing, it's just for illustration) is that every code change needs to be discussed on the mailing list. Now clearly that is absurd, so there has to be some threshold where discussing in a GH issue/PR or Slack is fine.
Is the name of the folder which we keep our source code in the repo important? No, it’s just above a bike shed color argument at best. If someone wants to discuss it on the list, go for it. In my mind, if someone doesn’t and wants to discuss it on a PR and gets enough feedback there: that is also fine to me. The PR speaks for itself, and it is not an important historical decision, and arguably: finding the reasoning when it is done in a GH pr is easier than having to search the mailing list archive, both are equally as permanent.
So I’m not sure what you are actually proposing, since we already do make important decisions on the list, and that discussing the minutia of every decision isn’t practical.
-ash
Re: [DISCUSS] Decisions made on devlist
Posted by Jarek Potiuk <ja...@potiuk.com>.
OK. Let me clarify then,
1) First of all - yes - i completely missed cadwyn and API versioning we
agreed to in the AIP. We had so many of those AIPs that I simply forgot
this tiny-little detail. Totally my fault and sorry for calling that one
out. Sorry Ash for that if you felt you've been called out needlessly.
2) Secondly - I am amazed how well we are delivering Airflow 3 together.
This an amazing feat to deliver it with such a speed, and coordination and
the way how we work together - exactly via combination of those different,
well balanced communication methods:
* having a grand plan and timeline that Vikram and Kaxil are keeping
updating and we synchronize and plan our deliveries on a weekly basis
according to Airflow 3 plan
https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+3.0+Development+Milestones
* earlier agreeing and discussing HUGE impact decisions in AIPs
* making decisions on things impacting the whole project and all people in
DEVLIST
* making quick decisions and discussions in PRs where particular problems
are solved and need a discussion from involved people
* quick discussions on slack where things need to be decided and reacted
on-the-spot
This all works wonderfully. I would even say miraculously in the
distributed environment of ours, where we a) work remotely, b) in
disconnected teams without hierarchy c) each of us focuses on their part
while following what others are doing and trying to fit-in
This is - and I am not afraid to say it - an ultimate proof where
"community over code" is showing its real power, where various stakeholders
are involved and we all work in one direction, despite each of us working
separately. Chapeau bax to everyone involved - everyone puts their own
efforts as a separate brick and we build the building together, and -
surprisingly the new building we build looks already fantastic and very
coherent.
In this context, a little more explanation on the reasoning here. I just
wanted to make sure that the "packaging' and "repository layout" comes into
the picture. The thread here was sparked with a slack discussion where TP
commented essentially "we should change our layout of folder, I do not like
the lack of consistency" - in a random slack conversation, that many of us
could have missed. And I was called out essentially for saying "We've been
discussing it for weeks on the devlist, please bring it to devlist and
let's discuss it there".
When we talk about packaging and connected layout of our repository has
been "crossing" all that - aiming to improve "contributor's journey" and
supporting the new architecture by splitting airflow into actually
independent pieces, I personally very carefully planned timeline and
delivery of multi-step packaging changes to fit-in and to put my own brick
where it fits (and exactly when it was planned according to the milestones
and a grand plan for Airflow 3 timeline we agreed to.
Initially we were agreeing high level changes with - slack discussions and
general ideas, some discussions with Ash and few other people who were
involved in packaging, Heck - it was not even only me - Ash was initially
separating providers as a whole, then added `task_sdk", eventually last
week we agreed to the new "airflow-ctl" separation as something that has
been moved out of Airflow 3.0 - so Jed and Bugra did it "quickly" to fit in
the beta timeline, so yeah that was done "quick-and-dirty" way now to
enable Airflow 3 timeline. In the meantime I spent a lot of time preparing
and testing and communicating all the plans to make the biggest and
potentially most disruptive move - moving all airflow code.
It's been many weeks in preparations, discussions in slack, earlier
provider's separation was treated as a "playground" where we solved a
number of teething issues, learned how uv works for us, spend a lot of time
on teaching people how to work with it, documenting and trying to
communicate on all available channels - slack, devlist, devcall demos etc.
on what those changes are going to bring. We discussed (on devlist this
time) the new structure, proposed and agreed to on various names, and this
HUGE move (I think the biggest PR ever with ~ 3000 files moved) was a
result of all that preparatory work and countless hours of testing and
making things work after the move. It's not as easy as "move stuff to a new
directory". Ash knows it best when he moved the "providers" to a new
"providers" folder that this is a LOT of effort - mainly because our test
harness that we use daily is heavily affected by such a move.
At all the stages when we knew (everyone involved in such moves) - the how
and what has been announced and discussed in the devlist (and slack) weeks
in advance to prepare everyone to what's coming and to get feedback - and a
lot of feedback from multiple people involved we had and applied. All was
good.
Until suddenly, after all those discussions, agreement, tests,
communication, right at the start of the busiest period of testing for RC
candidate where packaging was supposed to be done exactly **now** according
to the plans that we all agreed and followed, when I hear "I do not like
the new layout, please change" my natural reaction is "bring it to
devlist". We need to discuss it, realise the consequences, understand why
want to make such a - again - potentially hugely disruptive and costly
change, and first of all we need to understand if this is the right time to
again disrupt the work of pretty much everyone working on Airflow 3 by
changing the layout of our repo (which I'd estimate as again at least few
weeks of preparation, communication of work) is the right thing to do now.
I just think that "bringing this to a devlist" in this context rather than
"let's discuss it in this random slack thread" was not too outrageous of a
proposal? I am not sure if I did something wrong by proposing it to TP? Or
maybe we have a different understanding of what should and what should not
be discussed on the devlist?
This question of mine here was mostly - do I understand the way we
communicate here wrongly? Should we really agree and discuss such hugely
impactful and heavy changes on slack? Is that wrong to ask someone to bring
such a discussion to devlist?
Those are my questions really.
J.
On Sat, Mar 22, 2025 at 8:37 PM Vikram Koka <vi...@astronomer.io.invalid>
wrote:
> I completely agree regarding transparency, having options expressed in
> writing, and tracking decisions.
>
> I also support the ASF philosophy that voting should only happen on the dev
> list.
>
> However, treating the dev list as the only acceptable mechanism for
> discussion feels overly burdensome, in my humble opinion.
> In practice, I don't believe that email is not structured enough for all
> types of interactions.
> Here are two simple examples:
> 1. *Github issues*
> We have a template for reporting issues that enforces structure - capturing
> key elements such as Airflow version, affected area, reproduction steps,
> and so on. This level of enforced structure is hard to achieve in email.
>
> We also frequently respond to Github issues with decisions - explicitly or
> implicitly - which, in an abstract sense, are still decisions.
>
> 2. *AIPs (Airflow Improvement Proposals)*
> We have a template for AIPs. While many improvement ideas start as a simple
> thought in a Github issue or email, we often ask contributors to write them
> up as AIPs. The structure helps guide discussion, enables threaded
> comments, and supports versioning. Personally, I find the AIP process quite
> helpful.
> We do reference the AIP in the voting thread on the dev list, but the bulk
> of the discussion happens in Confluence, within the context of the written
> document.
>
> I may be misunderstanding the intent of this discussion, but one of the
> referenced examples - specifically "the versioning rules for the API" - was
> already covered in the original AIP
> <
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-72+Task+Execution+Interface+aka+Task+SDK
> >,
> in the following paragraph:
> *"This API should be strongly versioned using CalVer, and we propose an
> approach such as https://docs.cadwyn.dev/ <https://docs.cadwyn.dev/> to
> allow us to easily support many versions in parallel (tl;dr of this
> project: calver + request "migrations" lets you easily handle multiple
> versions API clients without needing parallel support, even what might
> otherwise be a breaking change, as long as change from one version to the
> next does not throw away information.) Each change, no matter how small, to
> this API will result in a new CalVer."*
>
> I have always viewed ASF processes as encouraging written, open
> communication to foster transparency in decision making, but not as
> precluding the use of openly accessible applications such as Github and
> Wiki.
>
> So far, I believe the Github and AIP processes have worked well alongside
> the dev list, and I am unsure what the motivation is for requiring a change
> here.
>
> Vikram
>
>
> On Sat, Mar 22, 2025 at 10:20 AM Michał Modras
> <mi...@google.com.invalid> wrote:
>
> > +1, I fully agree with the mechanism - it is open, transparent, gives
> > everyone an opportunity to participate, and keeps track of how the
> > decisions are made.
> >
> > On Sat, Mar 22, 2025 at 5:34 PM Shahar Epstein <sh...@apache.org>
> wrote:
> >
> > > Overall I agree that decisions should be made in the devlist.
> > > One improvement that I'd like to suggest, though, is to have a
> dedicated
> > > page on cwiki that summarizes all recent and upcoming decisions (votes
> +
> > > lazy consensus) in a table, including links to their respective devlist
> > > threads. We could have a page for important discussions as well.
> > > By doing it, we'll gain:
> > > 1. Better transparency - instead of searching within many emails for
> > > past decisions in client/Apache mail archives, we'll just look them up
> at
> > > the table.
> > > 2. Better noticeability - people would be more easily aware of
> > > upcoming votes. Personally I often miss emails as I redirect them to a
> > > subdirectory, and I get disappointed if I miss important votes.
> > >
> > > We already attach links to threads for AIPs in their wiki page, but I
> > think
> > > that having a more generalized page would benefit us all.
> > > That's my two ₪ anyway :)
> > >
> > >
> > > Shahar
> > >
> > > On Sat, Mar 22, 2025 at 12:09 AM Jarek Potiuk <ja...@potiuk.com>
> wrote:
> > >
> > > > I also would like to start another thread regarding decision making
> in
> > > our
> > > > projects. We had some discussions about decision making in our
> project.
> > > >
> > > > As an ASF project we are supposed to make important decisions on the
> > > > devlist. Full stop. "If it did not happen at the devlist, it did not
> > > > happen" is something that you can hear often in the ASF - even if the
> > > only
> > > > place you can actually see it written this way is
> > > > https://www.apache.org/press/highlights.html#2017
> > > >
> > > > The Apache Software Foundation has a very clear notion of important
> > > > decision making:
> > > >
> > > > * official voting rules
> https://www.apache.org/foundation/voting.html
> > > > * community development guidelines
> > > > https://community.apache.org/committers/decisionMaking.html
> > > >
> > > > Both of those refer to decision making at the devlist.
> > > >
> > > > We had a few - unnecessarily heated - discussions on how decisions
> are
> > > made
> > > > in the project. I personally think that our decision making should
> be
> > > done
> > > > at the devlist - where anyone can participate. Things like versioning
> > > rules
> > > > for API (hinted by Ash at the last dev call) or how our repo is
> > > structured
> > > > (a new proposal raised today by TP) should - IMHO - be discussed
> here,
> > at
> > > > the devlist.
> > > >
> > > > Not in a slack thread, not in private discussion, not when two or
> more
> > > > people talk to each other in a call. It's fine to discuss things
> > outside
> > > -
> > > > of course, but then any proposals for a change of things that we
> > already
> > > > discussed for weeks and either implicitly (by non-objection) or
> > > explicitly
> > > > (by not responding to [LAZY CONSENSUS] or [VOTE] thread in the
> devlist)
> > > are
> > > > just discussions. If someone wants a change, starting a thread in
> > devlist
> > > > is the right way of proposing a change. Especially for things that
> were
> > > > discussed before - sometimes for weeks, and no concerns were raised.
> > > >
> > > > IMHO - it's very simple - want a change - start and lead a [DISCUSS].
> > > [LAZY
> > > > CONSENSUS]. or [VOTE] (depending on the level of disagreement and
> > number
> > > of
> > > > different opinions). Not very complex - it just requires to start and
> > > lead
> > > > a discussion mailing list thread. Super inclusive and follows the
> > > archiving
> > > > and all other requirements of the ASF (for example you do not have to
> > > agree
> > > > TOC of Slack to participate)
> > > >
> > > > I would love to hear if others disagree with it and think that this
> > > process
> > > > is overly complex or problematic. I think it's quite clear,
> reasonable
> > > and
> > > > great to keep community decision making in check and follow all the
> > rules
> > > > and expectations of the ASF, but maybe there are some concerns with
> the
> > > > process and we would like to improve it.
> > > >
> > > > I would love to hear what others think about it.
> > > >
> > > > J.
> > > >
> > >
> >
>
Re: [DISCUSS] Decisions made on devlist
Posted by Vikram Koka <vi...@astronomer.io.INVALID>.
I completely agree regarding transparency, having options expressed in
writing, and tracking decisions.
I also support the ASF philosophy that voting should only happen on the dev
list.
However, treating the dev list as the only acceptable mechanism for
discussion feels overly burdensome, in my humble opinion.
In practice, I don't believe that email is not structured enough for all
types of interactions.
Here are two simple examples:
1. *Github issues*
We have a template for reporting issues that enforces structure - capturing
key elements such as Airflow version, affected area, reproduction steps,
and so on. This level of enforced structure is hard to achieve in email.
We also frequently respond to Github issues with decisions - explicitly or
implicitly - which, in an abstract sense, are still decisions.
2. *AIPs (Airflow Improvement Proposals)*
We have a template for AIPs. While many improvement ideas start as a simple
thought in a Github issue or email, we often ask contributors to write them
up as AIPs. The structure helps guide discussion, enables threaded
comments, and supports versioning. Personally, I find the AIP process quite
helpful.
We do reference the AIP in the voting thread on the dev list, but the bulk
of the discussion happens in Confluence, within the context of the written
document.
I may be misunderstanding the intent of this discussion, but one of the
referenced examples - specifically "the versioning rules for the API" - was
already covered in the original AIP
<https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-72+Task+Execution+Interface+aka+Task+SDK>,
in the following paragraph:
*"This API should be strongly versioned using CalVer, and we propose an
approach such as https://docs.cadwyn.dev/ <https://docs.cadwyn.dev/> to
allow us to easily support many versions in parallel (tl;dr of this
project: calver + request "migrations" lets you easily handle multiple
versions API clients without needing parallel support, even what might
otherwise be a breaking change, as long as change from one version to the
next does not throw away information.) Each change, no matter how small, to
this API will result in a new CalVer."*
I have always viewed ASF processes as encouraging written, open
communication to foster transparency in decision making, but not as
precluding the use of openly accessible applications such as Github and
Wiki.
So far, I believe the Github and AIP processes have worked well alongside
the dev list, and I am unsure what the motivation is for requiring a change
here.
Vikram
On Sat, Mar 22, 2025 at 10:20 AM Michał Modras
<mi...@google.com.invalid> wrote:
> +1, I fully agree with the mechanism - it is open, transparent, gives
> everyone an opportunity to participate, and keeps track of how the
> decisions are made.
>
> On Sat, Mar 22, 2025 at 5:34 PM Shahar Epstein <sh...@apache.org> wrote:
>
> > Overall I agree that decisions should be made in the devlist.
> > One improvement that I'd like to suggest, though, is to have a dedicated
> > page on cwiki that summarizes all recent and upcoming decisions (votes +
> > lazy consensus) in a table, including links to their respective devlist
> > threads. We could have a page for important discussions as well.
> > By doing it, we'll gain:
> > 1. Better transparency - instead of searching within many emails for
> > past decisions in client/Apache mail archives, we'll just look them up at
> > the table.
> > 2. Better noticeability - people would be more easily aware of
> > upcoming votes. Personally I often miss emails as I redirect them to a
> > subdirectory, and I get disappointed if I miss important votes.
> >
> > We already attach links to threads for AIPs in their wiki page, but I
> think
> > that having a more generalized page would benefit us all.
> > That's my two ₪ anyway :)
> >
> >
> > Shahar
> >
> > On Sat, Mar 22, 2025 at 12:09 AM Jarek Potiuk <ja...@potiuk.com> wrote:
> >
> > > I also would like to start another thread regarding decision making in
> > our
> > > projects. We had some discussions about decision making in our project.
> > >
> > > As an ASF project we are supposed to make important decisions on the
> > > devlist. Full stop. "If it did not happen at the devlist, it did not
> > > happen" is something that you can hear often in the ASF - even if the
> > only
> > > place you can actually see it written this way is
> > > https://www.apache.org/press/highlights.html#2017
> > >
> > > The Apache Software Foundation has a very clear notion of important
> > > decision making:
> > >
> > > * official voting rules https://www.apache.org/foundation/voting.html
> > > * community development guidelines
> > > https://community.apache.org/committers/decisionMaking.html
> > >
> > > Both of those refer to decision making at the devlist.
> > >
> > > We had a few - unnecessarily heated - discussions on how decisions are
> > made
> > > in the project. I personally think that our decision making should be
> > done
> > > at the devlist - where anyone can participate. Things like versioning
> > rules
> > > for API (hinted by Ash at the last dev call) or how our repo is
> > structured
> > > (a new proposal raised today by TP) should - IMHO - be discussed here,
> at
> > > the devlist.
> > >
> > > Not in a slack thread, not in private discussion, not when two or more
> > > people talk to each other in a call. It's fine to discuss things
> outside
> > -
> > > of course, but then any proposals for a change of things that we
> already
> > > discussed for weeks and either implicitly (by non-objection) or
> > explicitly
> > > (by not responding to [LAZY CONSENSUS] or [VOTE] thread in the devlist)
> > are
> > > just discussions. If someone wants a change, starting a thread in
> devlist
> > > is the right way of proposing a change. Especially for things that were
> > > discussed before - sometimes for weeks, and no concerns were raised.
> > >
> > > IMHO - it's very simple - want a change - start and lead a [DISCUSS].
> > [LAZY
> > > CONSENSUS]. or [VOTE] (depending on the level of disagreement and
> number
> > of
> > > different opinions). Not very complex - it just requires to start and
> > lead
> > > a discussion mailing list thread. Super inclusive and follows the
> > archiving
> > > and all other requirements of the ASF (for example you do not have to
> > agree
> > > TOC of Slack to participate)
> > >
> > > I would love to hear if others disagree with it and think that this
> > process
> > > is overly complex or problematic. I think it's quite clear, reasonable
> > and
> > > great to keep community decision making in check and follow all the
> rules
> > > and expectations of the ASF, but maybe there are some concerns with the
> > > process and we would like to improve it.
> > >
> > > I would love to hear what others think about it.
> > >
> > > J.
> > >
> >
>
Re: [DISCUSS] Decisions made on devlist
Posted by Michał Modras <mi...@google.com.INVALID>.
+1, I fully agree with the mechanism - it is open, transparent, gives
everyone an opportunity to participate, and keeps track of how the
decisions are made.
On Sat, Mar 22, 2025 at 5:34 PM Shahar Epstein <sh...@apache.org> wrote:
> Overall I agree that decisions should be made in the devlist.
> One improvement that I'd like to suggest, though, is to have a dedicated
> page on cwiki that summarizes all recent and upcoming decisions (votes +
> lazy consensus) in a table, including links to their respective devlist
> threads. We could have a page for important discussions as well.
> By doing it, we'll gain:
> 1. Better transparency - instead of searching within many emails for
> past decisions in client/Apache mail archives, we'll just look them up at
> the table.
> 2. Better noticeability - people would be more easily aware of
> upcoming votes. Personally I often miss emails as I redirect them to a
> subdirectory, and I get disappointed if I miss important votes.
>
> We already attach links to threads for AIPs in their wiki page, but I think
> that having a more generalized page would benefit us all.
> That's my two ₪ anyway :)
>
>
> Shahar
>
> On Sat, Mar 22, 2025 at 12:09 AM Jarek Potiuk <ja...@potiuk.com> wrote:
>
> > I also would like to start another thread regarding decision making in
> our
> > projects. We had some discussions about decision making in our project.
> >
> > As an ASF project we are supposed to make important decisions on the
> > devlist. Full stop. "If it did not happen at the devlist, it did not
> > happen" is something that you can hear often in the ASF - even if the
> only
> > place you can actually see it written this way is
> > https://www.apache.org/press/highlights.html#2017
> >
> > The Apache Software Foundation has a very clear notion of important
> > decision making:
> >
> > * official voting rules https://www.apache.org/foundation/voting.html
> > * community development guidelines
> > https://community.apache.org/committers/decisionMaking.html
> >
> > Both of those refer to decision making at the devlist.
> >
> > We had a few - unnecessarily heated - discussions on how decisions are
> made
> > in the project. I personally think that our decision making should be
> done
> > at the devlist - where anyone can participate. Things like versioning
> rules
> > for API (hinted by Ash at the last dev call) or how our repo is
> structured
> > (a new proposal raised today by TP) should - IMHO - be discussed here, at
> > the devlist.
> >
> > Not in a slack thread, not in private discussion, not when two or more
> > people talk to each other in a call. It's fine to discuss things outside
> -
> > of course, but then any proposals for a change of things that we already
> > discussed for weeks and either implicitly (by non-objection) or
> explicitly
> > (by not responding to [LAZY CONSENSUS] or [VOTE] thread in the devlist)
> are
> > just discussions. If someone wants a change, starting a thread in devlist
> > is the right way of proposing a change. Especially for things that were
> > discussed before - sometimes for weeks, and no concerns were raised.
> >
> > IMHO - it's very simple - want a change - start and lead a [DISCUSS].
> [LAZY
> > CONSENSUS]. or [VOTE] (depending on the level of disagreement and number
> of
> > different opinions). Not very complex - it just requires to start and
> lead
> > a discussion mailing list thread. Super inclusive and follows the
> archiving
> > and all other requirements of the ASF (for example you do not have to
> agree
> > TOC of Slack to participate)
> >
> > I would love to hear if others disagree with it and think that this
> process
> > is overly complex or problematic. I think it's quite clear, reasonable
> and
> > great to keep community decision making in check and follow all the rules
> > and expectations of the ASF, but maybe there are some concerns with the
> > process and we would like to improve it.
> >
> > I would love to hear what others think about it.
> >
> > J.
> >
>
Re: [DISCUSS] Decisions made on devlist
Posted by Shahar Epstein <sh...@apache.org>.
Overall I agree that decisions should be made in the devlist.
One improvement that I'd like to suggest, though, is to have a dedicated
page on cwiki that summarizes all recent and upcoming decisions (votes +
lazy consensus) in a table, including links to their respective devlist
threads. We could have a page for important discussions as well.
By doing it, we'll gain:
1. Better transparency - instead of searching within many emails for
past decisions in client/Apache mail archives, we'll just look them up at
the table.
2. Better noticeability - people would be more easily aware of
upcoming votes. Personally I often miss emails as I redirect them to a
subdirectory, and I get disappointed if I miss important votes.
We already attach links to threads for AIPs in their wiki page, but I think
that having a more generalized page would benefit us all.
That's my two ₪ anyway :)
Shahar
On Sat, Mar 22, 2025 at 12:09 AM Jarek Potiuk <ja...@potiuk.com> wrote:
> I also would like to start another thread regarding decision making in our
> projects. We had some discussions about decision making in our project.
>
> As an ASF project we are supposed to make important decisions on the
> devlist. Full stop. "If it did not happen at the devlist, it did not
> happen" is something that you can hear often in the ASF - even if the only
> place you can actually see it written this way is
> https://www.apache.org/press/highlights.html#2017
>
> The Apache Software Foundation has a very clear notion of important
> decision making:
>
> * official voting rules https://www.apache.org/foundation/voting.html
> * community development guidelines
> https://community.apache.org/committers/decisionMaking.html
>
> Both of those refer to decision making at the devlist.
>
> We had a few - unnecessarily heated - discussions on how decisions are made
> in the project. I personally think that our decision making should be done
> at the devlist - where anyone can participate. Things like versioning rules
> for API (hinted by Ash at the last dev call) or how our repo is structured
> (a new proposal raised today by TP) should - IMHO - be discussed here, at
> the devlist.
>
> Not in a slack thread, not in private discussion, not when two or more
> people talk to each other in a call. It's fine to discuss things outside -
> of course, but then any proposals for a change of things that we already
> discussed for weeks and either implicitly (by non-objection) or explicitly
> (by not responding to [LAZY CONSENSUS] or [VOTE] thread in the devlist) are
> just discussions. If someone wants a change, starting a thread in devlist
> is the right way of proposing a change. Especially for things that were
> discussed before - sometimes for weeks, and no concerns were raised.
>
> IMHO - it's very simple - want a change - start and lead a [DISCUSS]. [LAZY
> CONSENSUS]. or [VOTE] (depending on the level of disagreement and number of
> different opinions). Not very complex - it just requires to start and lead
> a discussion mailing list thread. Super inclusive and follows the archiving
> and all other requirements of the ASF (for example you do not have to agree
> TOC of Slack to participate)
>
> I would love to hear if others disagree with it and think that this process
> is overly complex or problematic. I think it's quite clear, reasonable and
> great to keep community decision making in check and follow all the rules
> and expectations of the ASF, but maybe there are some concerns with the
> process and we would like to improve it.
>
> I would love to hear what others think about it.
>
> J.
>