You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Mick Semb Wever <mc...@apache.org> on 2019/09/16 18:03:08 UTC
[DISCUSS] Proposing an Cassandra Enhancement Proposal (CEP) process
With the feature freeze for 4.0 getting a little closer to its end, and after Scott's NGCC presentation on how Cassandra can be better at moving forward, I'm keen to bring up the idea of a "Cassandra Enhancement Proposal" (CEP) process.
Big changes in the past have not always been as transparent and clear as they could have been in their initial stages. For example they have been written up in jira tickets, in a way that becomes quite difficult to unpack afterwards the difference between the initial proposal and the actual agreed upon design+implementation, or a worse example, ideas that were largely fleshed out behind closed (or hidden) doors and then only implemented in public.
A year ago a rough CEP (CIP) draft was put together, which was largely just a copy of what Kafka and Spark do. Now Scott has done a bit of work in formulating this into something that is higher-level and less design-document-heavy.
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652201
Are they edits to the CEP, or its template, that folk would like to see?
Are we ready for such a CEP process?
Can we just take the jump and request that all new feature tickets link to a CEP?
regards,
Mick
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Proposing an Cassandra Enhancement Proposal (CEP) process
Posted by Joshua McKenzie <jm...@apache.org>.
All too often, a work-invalidating insight hits late in a cycle while
people are talking about something and significant work has been done on
the invalidated proposal. A CEP up front with engagement from a bunch of
parties may very well help surface those design implications sooner, but we
also have a pretty old code-base with a lot of interesting edge-cases in it
(both in code and in state/domain) that we won't realize until we're in the
thick of it.
So +1 to bes' general sentiments here.
On Tue, Sep 17, 2019 at 1:53 PM Benedict Elliott Smith <be...@apache.org>
wrote:
> We have to be very careful here in my opinion. While the process may
> provide some moral authority, particularly on matters of taste or opinion,
> we cannot mandate participation, else accept the decisions that arise.
> People are legitimately busy, and have to steal their spare time to
> participate - which can be draining. Particularly for large undertakings,
> where context and headspace can be costly, even more so when you have your
> own projects to deliver.
>
> I only personally endorse giving the _benefit of the doubt_ to
> participants in a CEP/CIP. It cannot be carte blanche, or even a
> presumption of a decision being made in its favour. A CEP/CIP should still
> aim to bring the _most likely_ to be accepted proposal to the whole project
> for its endorsement, or for further revision.
>
> Our goal here should be to mitigate risk for all parties, without
> disrupting the governance of the project. Everyone should be
> _incentivised_ to behave in desirable way, but we should not penalise too
> heavily any individual (or the project, by ignoring them) for failing to do
> so.
>
> It seems to follow that a full PMC vote could be held to accept or reject
> the result of any CEP/CIP, rather than the normal +1/-1 of a single
> committer/PMC member. I would favour this meaning there no veto is
> possible at this stage, which would provide a further incentive to
> authors. In this case we only need to agree guidance for how a completed
> CEP/CIP should be received for such a vote, since we cannot bind a future
> PMC anyway.
>
>
> On 17/09/2019, 18:19, "sankalp kohli" <ko...@gmail.com> wrote:
>
> Another thing which it should solve is someone proposing an alternate
> very
> late into development which could be provided sooner. If someone has a
> good
> feedback which could not have been given at the time of CEP then that
> is
> good. We don't want situations where contributors have done the CEP and
> then worked on implementation of it and then someone who has not read
> the
> CEP comes in and starts giving feedback. This feedback should come at
> the
> time of CEP if CEP has covered that area.
> To be clear, I am not saying people should not give feedback later just
> that they dont ignore the whole thing and wake up later in the process.
> This causes huge productivity and morale loss to code contributors who
> are
> in minority right now in the community.
>
> On Tue, Sep 17, 2019 at 3:44 AM Benedict Elliott Smith <
> benedict@apache.org>
> wrote:
>
> > Can we modify the document to make this really explicit then? Right
> now,
> > the language suggests the process is mandated, rather than
> encouraged and
> > beneficial.
> >
> > It would be nice to frame it as a positive and incentivised
> undertaking by
> > authors, and to list the intended advantages, as well as the
> potential
> > disadvantages of not undertaking it, while making it clear it is left
> > entirely to their own judgement whether or not to do so.
> >
> > To be really clear, I do not refer to the flexible definition of the
> > process, but to whether participation in even a modest
> interpretation of
> > the process is necessary at all. This is a form of pre-registration
> for
> > work, to achieve community buy-in. If you want to go ahead and do
> > something on your own, you only risk difficulty and delays when
> obtaining
> > community buy-in after the fact. Let's not dissuade hobbyists,
> part-timers
> > or scratching an itch by suggesting their work will be discounted
> because
> > it wasn't pre-registered.
> >
> >
> > On 17/09/2019, 06:46, "Mick Semb Wever" <mc...@apache.org> wrote:
> >
> >
> >
> > > I think we need to have a meta discussion about the goal for
> > > introducing a new process.
> >
> >
> > Indeed, and these were only two brief examples that came to me.
> > Another, using the sidecar proposal as an example, is simply to
> ensure a
> > little patience is taken during the initial brainstorming and
> navigation
> > phase, to give more open collaboration a better chance. What's in the
> > landscape, where's the value, who might be interested in getting
> involved
> > in this, etc etc. I think the C* community has typically been pretty
> > amazing at this, but it would be nice to see it formalised a bit
> better.
> >
> >
> > > By not mandating it, we do not need to define where it is
> necessary;
> > > the larger and more impactful the change, the greater the
> incentive
> > to
> > > the author.
> >
> >
> > This is what Scott highlighted well.
> > Sure, a CEP could be opened with nothing but a title to begin
> with.
> > And where it goes from there is up to the working group that
> materialises.
> > Just to have a landing space for new features that's not Jira, I
> believe
> > would be of value.
> >
> > And in no way should the CEP be a return to waterfall. As you
> say,
> > late discoveries and feedback (as annoying as it can be) is all part
> of the
> > agile game.
> >
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> > For additional commands, e-mail: dev-help@cassandra.apache.org
> >
> >
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] Proposing an Cassandra Enhancement Proposal (CEP)
process
Posted by Benedict Elliott Smith <be...@apache.org>.
We have to be very careful here in my opinion. While the process may provide some moral authority, particularly on matters of taste or opinion, we cannot mandate participation, else accept the decisions that arise. People are legitimately busy, and have to steal their spare time to participate - which can be draining. Particularly for large undertakings, where context and headspace can be costly, even more so when you have your own projects to deliver.
I only personally endorse giving the _benefit of the doubt_ to participants in a CEP/CIP. It cannot be carte blanche, or even a presumption of a decision being made in its favour. A CEP/CIP should still aim to bring the _most likely_ to be accepted proposal to the whole project for its endorsement, or for further revision.
Our goal here should be to mitigate risk for all parties, without disrupting the governance of the project. Everyone should be _incentivised_ to behave in desirable way, but we should not penalise too heavily any individual (or the project, by ignoring them) for failing to do so.
It seems to follow that a full PMC vote could be held to accept or reject the result of any CEP/CIP, rather than the normal +1/-1 of a single committer/PMC member. I would favour this meaning there no veto is possible at this stage, which would provide a further incentive to authors. In this case we only need to agree guidance for how a completed CEP/CIP should be received for such a vote, since we cannot bind a future PMC anyway.
On 17/09/2019, 18:19, "sankalp kohli" <ko...@gmail.com> wrote:
Another thing which it should solve is someone proposing an alternate very
late into development which could be provided sooner. If someone has a good
feedback which could not have been given at the time of CEP then that is
good. We don't want situations where contributors have done the CEP and
then worked on implementation of it and then someone who has not read the
CEP comes in and starts giving feedback. This feedback should come at the
time of CEP if CEP has covered that area.
To be clear, I am not saying people should not give feedback later just
that they dont ignore the whole thing and wake up later in the process.
This causes huge productivity and morale loss to code contributors who are
in minority right now in the community.
On Tue, Sep 17, 2019 at 3:44 AM Benedict Elliott Smith <be...@apache.org>
wrote:
> Can we modify the document to make this really explicit then? Right now,
> the language suggests the process is mandated, rather than encouraged and
> beneficial.
>
> It would be nice to frame it as a positive and incentivised undertaking by
> authors, and to list the intended advantages, as well as the potential
> disadvantages of not undertaking it, while making it clear it is left
> entirely to their own judgement whether or not to do so.
>
> To be really clear, I do not refer to the flexible definition of the
> process, but to whether participation in even a modest interpretation of
> the process is necessary at all. This is a form of pre-registration for
> work, to achieve community buy-in. If you want to go ahead and do
> something on your own, you only risk difficulty and delays when obtaining
> community buy-in after the fact. Let's not dissuade hobbyists, part-timers
> or scratching an itch by suggesting their work will be discounted because
> it wasn't pre-registered.
>
>
> On 17/09/2019, 06:46, "Mick Semb Wever" <mc...@apache.org> wrote:
>
>
>
> > I think we need to have a meta discussion about the goal for
> > introducing a new process.
>
>
> Indeed, and these were only two brief examples that came to me.
> Another, using the sidecar proposal as an example, is simply to ensure a
> little patience is taken during the initial brainstorming and navigation
> phase, to give more open collaboration a better chance. What's in the
> landscape, where's the value, who might be interested in getting involved
> in this, etc etc. I think the C* community has typically been pretty
> amazing at this, but it would be nice to see it formalised a bit better.
>
>
> > By not mandating it, we do not need to define where it is necessary;
> > the larger and more impactful the change, the greater the incentive
> to
> > the author.
>
>
> This is what Scott highlighted well.
> Sure, a CEP could be opened with nothing but a title to begin with.
> And where it goes from there is up to the working group that materialises.
> Just to have a landing space for new features that's not Jira, I believe
> would be of value.
>
> And in no way should the CEP be a return to waterfall. As you say,
> late discoveries and feedback (as annoying as it can be) is all part of the
> agile game.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Proposing an Cassandra Enhancement Proposal (CEP) process
Posted by sankalp kohli <ko...@gmail.com>.
Another thing which it should solve is someone proposing an alternate very
late into development which could be provided sooner. If someone has a good
feedback which could not have been given at the time of CEP then that is
good. We don't want situations where contributors have done the CEP and
then worked on implementation of it and then someone who has not read the
CEP comes in and starts giving feedback. This feedback should come at the
time of CEP if CEP has covered that area.
To be clear, I am not saying people should not give feedback later just
that they dont ignore the whole thing and wake up later in the process.
This causes huge productivity and morale loss to code contributors who are
in minority right now in the community.
On Tue, Sep 17, 2019 at 3:44 AM Benedict Elliott Smith <be...@apache.org>
wrote:
> Can we modify the document to make this really explicit then? Right now,
> the language suggests the process is mandated, rather than encouraged and
> beneficial.
>
> It would be nice to frame it as a positive and incentivised undertaking by
> authors, and to list the intended advantages, as well as the potential
> disadvantages of not undertaking it, while making it clear it is left
> entirely to their own judgement whether or not to do so.
>
> To be really clear, I do not refer to the flexible definition of the
> process, but to whether participation in even a modest interpretation of
> the process is necessary at all. This is a form of pre-registration for
> work, to achieve community buy-in. If you want to go ahead and do
> something on your own, you only risk difficulty and delays when obtaining
> community buy-in after the fact. Let's not dissuade hobbyists, part-timers
> or scratching an itch by suggesting their work will be discounted because
> it wasn't pre-registered.
>
>
> On 17/09/2019, 06:46, "Mick Semb Wever" <mc...@apache.org> wrote:
>
>
>
> > I think we need to have a meta discussion about the goal for
> > introducing a new process.
>
>
> Indeed, and these were only two brief examples that came to me.
> Another, using the sidecar proposal as an example, is simply to ensure a
> little patience is taken during the initial brainstorming and navigation
> phase, to give more open collaboration a better chance. What's in the
> landscape, where's the value, who might be interested in getting involved
> in this, etc etc. I think the C* community has typically been pretty
> amazing at this, but it would be nice to see it formalised a bit better.
>
>
> > By not mandating it, we do not need to define where it is necessary;
> > the larger and more impactful the change, the greater the incentive
> to
> > the author.
>
>
> This is what Scott highlighted well.
> Sure, a CEP could be opened with nothing but a title to begin with.
> And where it goes from there is up to the working group that materialises.
> Just to have a landing space for new features that's not Jira, I believe
> would be of value.
>
> And in no way should the CEP be a return to waterfall. As you say,
> late discoveries and feedback (as annoying as it can be) is all part of the
> agile game.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: dev-help@cassandra.apache.org
>
>
Re: [DISCUSS] Proposing an Cassandra Enhancement Proposal (CEP) process
Posted by Mick Semb Wever <mc...@apache.org>.
> The 'Purpose' section has
> been updated to meet your input around the flexibility of participation
> (and process). I think it is what you are after, but ofc I could be
> wrong (or it doesn't go far enough?). Feel free to edit the doc as well.
The "Cassandra Enhancement Proposal" document has gone through further edits, thanks Benedict, and I believe it is in a good place now for the community.
I will raise it to a vote next week, if there is no further objections or input, to establish accepting it as a formal document in the project.
regards,
Mick
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Proposing an Cassandra Enhancement Proposal (CEP) process
Posted by Mick Semb Wever <mc...@apache.org>.
> To be really clear, I do not refer to the flexible definition of the
> process, but to whether participation in even a modest interpretation
> of the process is necessary at all.
Benedict, could you check the document now. The 'Purpose' section has been updated to meet your input around the flexibility of participation (and process). I think it is what you are after, but ofc I could be wrong (or it doesn't go far enough?). Feel free to edit the doc as well.
I also put in (often in verbatim) some of the input from Sankalp and Joshua.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Proposing an Cassandra Enhancement Proposal (CEP)
process
Posted by Benedict Elliott Smith <be...@apache.org>.
Can we modify the document to make this really explicit then? Right now, the language suggests the process is mandated, rather than encouraged and beneficial.
It would be nice to frame it as a positive and incentivised undertaking by authors, and to list the intended advantages, as well as the potential disadvantages of not undertaking it, while making it clear it is left entirely to their own judgement whether or not to do so.
To be really clear, I do not refer to the flexible definition of the process, but to whether participation in even a modest interpretation of the process is necessary at all. This is a form of pre-registration for work, to achieve community buy-in. If you want to go ahead and do something on your own, you only risk difficulty and delays when obtaining community buy-in after the fact. Let's not dissuade hobbyists, part-timers or scratching an itch by suggesting their work will be discounted because it wasn't pre-registered.
On 17/09/2019, 06:46, "Mick Semb Wever" <mc...@apache.org> wrote:
> I think we need to have a meta discussion about the goal for
> introducing a new process.
Indeed, and these were only two brief examples that came to me. Another, using the sidecar proposal as an example, is simply to ensure a little patience is taken during the initial brainstorming and navigation phase, to give more open collaboration a better chance. What's in the landscape, where's the value, who might be interested in getting involved in this, etc etc. I think the C* community has typically been pretty amazing at this, but it would be nice to see it formalised a bit better.
> By not mandating it, we do not need to define where it is necessary;
> the larger and more impactful the change, the greater the incentive to
> the author.
This is what Scott highlighted well.
Sure, a CEP could be opened with nothing but a title to begin with. And where it goes from there is up to the working group that materialises. Just to have a landing space for new features that's not Jira, I believe would be of value.
And in no way should the CEP be a return to waterfall. As you say, late discoveries and feedback (as annoying as it can be) is all part of the agile game.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Proposing an Cassandra Enhancement Proposal (CEP) process
Posted by Mick Semb Wever <mc...@apache.org>.
> I think we need to have a meta discussion about the goal for
> introducing a new process.
Indeed, and these were only two brief examples that came to me. Another, using the sidecar proposal as an example, is simply to ensure a little patience is taken during the initial brainstorming and navigation phase, to give more open collaboration a better chance. What's in the landscape, where's the value, who might be interested in getting involved in this, etc etc. I think the C* community has typically been pretty amazing at this, but it would be nice to see it formalised a bit better.
> By not mandating it, we do not need to define where it is necessary;
> the larger and more impactful the change, the greater the incentive to
> the author.
This is what Scott highlighted well.
Sure, a CEP could be opened with nothing but a title to begin with. And where it goes from there is up to the working group that materialises. Just to have a landing space for new features that's not Jira, I believe would be of value.
And in no way should the CEP be a return to waterfall. As you say, late discoveries and feedback (as annoying as it can be) is all part of the agile game.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
Re: [DISCUSS] Proposing an Cassandra Enhancement Proposal (CEP)
process
Posted by Benedict Elliott Smith <be...@apache.org>.
I think we need to have a meta discussion about the goal for introducing a new process. Your email mentions two reasons that I can see:
1) Clarity of the outcome? "For example they have been written up in jira tickets, in a way that becomes quite difficult to unpack afterwards the difference between the initial proposal and the actual agreed upon design+implementation"
2) Community buy-in/input? "ideas that were largely fleshed out behind closed (or hidden) doors and then only implemented in public"
The process seems orthogonal to (1) - this is a matter of imposing stricter standards on docs before release, which we should mandate independently.
As to point (2), as formulated today ("When in doubt, if a committer thinks a change needs an CEP, it does."), a "CEP" may be demanded by anybody with a commit bit. It is problematic to me to ever _require_ this process, under any circumstances, because it is unnecessarily restrictive about how members of the community self-organise. It should be a voluntary process with clear benefits to the participants to incentivise them.
(It's also not at all clear how you impose it retroactively - what if a contributor didn't follow the processs, but has work to contribute?)
The contract should be simple: follow the process if you want progressive feedback on your work, and a high chance of it being accepted without major conceptual revisions at the final hurdle*. Otherwise, you risk the community rejecting your work.
By not mandating it, we do not need to define where it is necessary; the larger and more impactful the change, the greater the incentive to the author.
On a practical note, we should also be honest with ourselves, and others, that this is experimental. Given the overall involvement of people on Jira and the lists, I personally doubt any proposal will receive enough feedback during its progress to materially mitigate the risk that members of the community chime in at the final hurdle. If their concerns are valid, we have to listen to them, even then. We cannot mandate that people participate incrementally in others' work, because they have their own work to be doing, and if we curtail these contributors' ability to provide feedback later, the project will only be the poorer for it.
*The process may also provide legitimacy to decisions taken, so that matters primarily of preference _may_ be decided in favour of participants if later interlopers take exception.
On 16/09/2019, 19:03, "Mick Semb Wever" <mc...@apache.org> wrote:
With the feature freeze for 4.0 getting a little closer to its end, and after Scott's NGCC presentation on how Cassandra can be better at moving forward, I'm keen to bring up the idea of a "Cassandra Enhancement Proposal" (CEP) process.
Big changes in the past have not always been as transparent and clear as they could have been in their initial stages. For example they have been written up in jira tickets, in a way that becomes quite difficult to unpack afterwards the difference between the initial proposal and the actual agreed upon design+implementation, or a worse example, ideas that were largely fleshed out behind closed (or hidden) doors and then only implemented in public.
A year ago a rough CEP (CIP) draft was put together, which was largely just a copy of what Kafka and Spark do. Now Scott has done a bit of work in formulating this into something that is higher-level and less design-document-heavy.
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652201
Are they edits to the CEP, or its template, that folk would like to see?
Are we ready for such a CEP process?
Can we just take the jump and request that all new feature tickets link to a CEP?
regards,
Mick
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cassandra.apache.org
For additional commands, e-mail: dev-help@cassandra.apache.org