scaling proposals

[Benjamin Hindman <be...@gmail.com>]

The good news is, the project continues to grow! The bad news is, not all
of our procedures scale. I'd like to propose some changes to streamline
hotspots around the project.


*(1) Review Shepherds*

Companies that rely on Mesos expect it to be the foundation of their
software infrastructure and it's imperative that we ship high-quality and
robust code. To help facilitate this we put our code through rigorous
reviews. Unfortunately, this can often act as a bottleneck, especially when
nobody wants (or has time) to review your code!

I'd like us to be more accountable; I'd like to propose that all
significant code changes get shepherded by a PMC/committer as early in the
development phase as possible. We've played around with "review shepherds"
in the past and IMHO it's helped tremendously (and the earlier the
shepherding the better).

Here's how I'm envisioning this would look:

A contributor (or committer) would tell people they're interested in
working on a particular JIRA issue, feature, bug fix, TODO in the code,
etc. by either emailing dev@mesos.apache.org or posting a comment on JIRA.
Their note would specifically seek out a PMC/committer to act as a
shepherd. Note that the goal here is really to find a shepherd _before_ you
start architecting or coding!

It's possible that nobody will volunteer as a shepherd either because (a)
nobody has time due to prioritizing other things in the project or (b)
they're a new PMC/committer and don't yet feel comfortable shepherding.
Seeking a shepherd early is exactly meant to deal with issues around (a)
which I'll discuss in more detail below. In the case of (b), I'd like to
propose people "pair" shepherd. That is, a newer PMC/committer actively
find an older (in project years) PMC/committer.

To be clear, I don't think that all code changes should require a shepherd,
only "significant" ones. For now, I'd prefer we error on the side of
caution and seek out shepherds for most things, letting the shepherd decide
whether or not they believe the work requires them. In addition, I think we
should leave it up to the shepherd's best judgement to decide when design
documents or greater consensus around a certain change should be sought.

*How can you help!? *In order for this to work we'll need to actively guide
people towards finding a shepherd. Moreover, we'll need to set good
examples ourselves. People often snoop on project mailing lists and mimic
the behavior of those they observe.


*(2) Active/Desired*

One of the biggest reasons reviews go stagnant is because people just don't
have time to help review them. Often times this is an artifact of people
picking features to work on that are low in the priority list of the
project. To *help guide people* towards issues that are *desired* I'd like
to propose that we add a new JIRA labels called, drum roll please: desired.
In conjunction with the 'desired' label I'd like to propose we also add an
'active' label.

An active JIRA issue is something that a contributor/committer/organization
is actively working on (or has publicly allocated time to work on in the
next quarter). A desired JIRA issue is something that the
committers/organizations of the project would be working on if they had
more time! That is, things that the project community believes is of value
to the project and should get worked on.

An advantage of labeling issues this way is that it makes creating a
"dashboard" for the project relatively easy. In fact, Chris Lambert has
already prepared one here:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=33 (note:
that dashboard includes the 'newbie' label because these are also "desired"
issues just of smaller scope meant for new contributors). This dashboard
can help act as basis for a roadmap for the project as well.

To help triage what issues should be made desired I'd like to suggest we
start (a) voting on tickets and recommending others vote on tickets and (b)
encourage people to make desired things known by emailing the dev@ and
user@mailing lists. In the short term I'd like to help facilitate the
triaging
via emails to the list where I can gather feedback and label tickets as
appropriate. In the long term I'd love to evolve this into
monthly/bi-weekly community meetings where people have a chance to curate
desired issues.


*(3) Becoming a PMC Member / Committer*

Just like code, I'd like us to be accountable for growing the PMC/committer
base. Ultimately this will give us even more shepherds enabling the project
to handle even more concurrent changes. To do this effectively I'd like to
propose that we introduce shepherds for helping contributors become
committers (sorry for the overloaded user of the word shepherd!). Like code
changes, I think we need to be more proactive about assigning a shepherd to
someone that is interested in becoming a PMC/committer on the project. This
shepherd can help identify things that the contributor should demonstrate
in order to be a successful PMC/committer after potentially soliciting
feedback from the existing PMC. My hope is that this will make the actual
PMC/committer vote more of a formality than anything else.


In summary, I'd like to propose:

Code/review shepherds.
The addition of 'active' and 'desired' JIRA labels.
PMC/committer shepherds.

I'm clearly a +1 for these and I'm looking forward to hearing from others.

Ben.

Re: scaling proposals

[Dominic Hamon <dh...@twopensource.com>]

On Mon, Apr 21, 2014 at 11:51 AM, Benjamin Hindman <
benjamin.hindman@gmail.com> wrote:

> The good news is, the project continues to grow! The bad news is, not all
> of our procedures scale. I'd like to propose some changes to streamline
> hotspots around the project.
>
>
> *(1) Review Shepherds*
>
> Companies that rely on Mesos expect it to be the foundation of their
> software infrastructure and it's imperative that we ship high-quality and
> robust code. To help facilitate this we put our code through rigorous
> reviews. Unfortunately, this can often act as a bottleneck, especially when
> nobody wants (or has time) to review your code!
>
> I'd like us to be more accountable; I'd like to propose that all
> significant code changes get shepherded by a PMC/committer as early in the
> development phase as possible. We've played around with "review shepherds"
> in the past and IMHO it's helped tremendously (and the earlier the
> shepherding the better).
>

I agree that shepherding makes a huge difference, but one other part of
this that Mesos hasn't yet had to deal with is splitting the knowledge
base. Ie, it's assumed right now that every committer is familiar with
every aspect of the code-base. It may be worth considering splitting that
up so that committers are responsible for subsystems. This allows the wider
developer community to know which committer to go to for best response,
and allows committers to specialise more.



>
> Here's how I'm envisioning this would look:
>
> A contributor (or committer) would tell people they're interested in
> working on a particular JIRA issue, feature, bug fix, TODO in the code,
> etc. by either emailing dev@mesos.apache.org or posting a comment on JIRA.
> Their note would specifically seek out a PMC/committer to act as a
> shepherd. Note that the goal here is really to find a shepherd _before_ you
> start architecting or coding!
>

Is it enough to mark the JIRA ticket as 'in progress'? We should be able
to track these transitions and make sure a shepherd is attached as soon as
possible.



>
> It's possible that nobody will volunteer as a shepherd either because (a)
> nobody has time due to prioritizing other things in the project or (b)
> they're a new PMC/committer and don't yet feel comfortable shepherding.
> Seeking a shepherd early is exactly meant to deal with issues around (a)
> which I'll discuss in more detail below. In the case of (b), I'd like to
> propose people "pair" shepherd. That is, a newer PMC/committer actively
> find an older (in project years) PMC/committer.
>
> To be clear, I don't think that all code changes should require a shepherd,
> only "significant" ones. For now, I'd prefer we error on the side of
> caution and seek out shepherds for most things, letting the shepherd decide
> whether or not they believe the work requires them. In addition, I think we
> should leave it up to the shepherd's best judgement to decide when design
> documents or greater consensus around a certain change should be sought.
>

Is there a danger here that smaller changes (specifically cleanups) will
be neglected as they don't have a shepherd attached?



>
> *How can you help!? *In order for this to work we'll need to actively guide
> people towards finding a shepherd. Moreover, we'll need to set good
> examples ourselves. People often snoop on project mailing lists and mimic
> the behavior of those they observe.
>
>
> *(2) Active/Desired*
>
> One of the biggest reasons reviews go stagnant is because people just don't
> have time to help review them. Often times this is an artifact of people
> picking features to work on that are low in the priority list of the
> project. To *help guide people* towards issues that are *desired* I'd like
> to propose that we add a new JIRA labels called, drum roll please: desired.
> In conjunction with the 'desired' label I'd like to propose we also add an
> 'active' label.
>

The best way for new developers to get involved is to pick up low priority
tickets. I'd hate to see the project stagnate because those newer
contributors' patches don't get attention. I think we need to be very
conscious about encouraging new contributors by actively prioritising their
patches for review above others. This will get them up to speed quickly and
will sow the seeds for more committers and potential shepherds down the
road. 



>
> An active JIRA issue is something that a contributor/committer/organization
> is actively working on (or has publicly allocated time to work on in the
> next quarter). A desired JIRA issue is something that the
> committers/organizations of the project would be working on if they had
> more time! That is, things that the project community believes is of value
> to the project and should get worked on.
>

How about an 'active' ticket is one that is marked 'in progress' and a
'desired' ticket is everything else. If we have JIRA tickets for things we
don't think are of value to the project, we should close them out. There's
no point in holding on to tickets that we don't believe should be worked on
at all.



>
> An advantage of labeling issues this way is that it makes creating a
> "dashboard" for the project relatively easy. In fact, Chris Lambert has
> already prepared one here:
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=33 (note:
> that dashboard includes the 'newbie' label because these are also "desired"
> issues just of smaller scope meant for new contributors). This dashboard
> can help act as basis for a roadmap for the project as well.
>
> To help triage what issues should be made desired I'd like to suggest we
> start (a) voting on tickets and recommending others vote on tickets and (b)
> encourage people to make desired things known by emailing the dev@ and
> user@mailing lists. In the short term I'd like to help facilitate the
> triaging
> via emails to the list where I can gather feedback and label tickets as
> appropriate. In the long term I'd love to evolve this into
> monthly/bi-weekly community meetings where people have a chance to curate
> desired issues.
>
>
> *(3) Becoming a PMC Member / Committer*
>
> Just like code, I'd like us to be accountable for growing the PMC/committer
> base. Ultimately this will give us even more shepherds enabling the project
> to handle even more concurrent changes. To do this effectively I'd like to
> propose that we introduce shepherds for helping contributors become
> committers (sorry for the overloaded user of the word shepherd!). Like code
> changes, I think we need to be more proactive about assigning a shepherd to
> someone that is interested in becoming a PMC/committer on the project. This
> shepherd can help identify things that the contributor should demonstrate
> in order to be a successful PMC/committer after potentially soliciting
> feedback from the existing PMC. My hope is that this will make the actual
> PMC/committer vote more of a formality than anything else.
>
>
> In summary, I'd like to propose:
>
> Code/review shepherds.
> The addition of 'active' and 'desired' JIRA labels.
> PMC/committer shepherds.
>
> I'm clearly a +1 for these and I'm looking forward to hearing from others.
>

Thanks for putting all this down.

I completely support the push for a better review process, however I have
seen efforts like this fail in the past due to adding too much process to
the, well, process. There needs to be a clear path to becoming a committer
and eventual shepherd (number of successful patches is often a good
guideline) and as little up-front learning curve as possible for new
contributors. Finding a committer/shepherd for a change should be a simple
task and knowing when a shepherd is required should also be something a new
contributor can readily understand.

One thing that is critical is documenting the
contribution/committer/shepherd process and roles as soon as they're clear
and pointing new contributors to that document.
 

>
> Ben.
>

Re: scaling proposals

[Tim St Clair <ts...@redhat.com>]

inline 

----- Original Message -----
> From: "Benjamin Hindman" <be...@eecs.berkeley.edu>
> To: "dev" <de...@mesos.apache.org>
> Sent: Friday, May 2, 2014 4:39:19 PM
> Subject: Re: scaling proposals
> 
> First, please accept my apologies for not circling back to this sooner.
> It's a high-priority issue for me so I'm glad I've gotten such thoughtful
> responses from everyone!
> 
> Next, let me more formally define what I mean by 'active' and 'desired'.
> Before I do that, however, I want to acknowledge that they probably aren't
> the best names, but let's return to that after elaborating on their purpose.
> 
> I'll start with 'desired': it is true that every JIRA ticket is indeed
> "desired". Those that are not should really be resolved/closed. But not all
> JIRA tickets are equal. In particular, some are more important than others,
> some have dependencies that must be done before they can be started, etc. A
> lot of this can be captured with existing JIRA mechanisms like priority and
> dependencies. But this can quickly get inconsistent unless we triage new
> tickets and monitor existing tickets very very closely.
> 
> Rather than require this constant triaging I wanted us to do more
> "periodic" triaging (or on-demand when we run out of "desired" issues) and
> capture the result of that via explicit labels in JIRA. That is, people
> know that issues with this label have been culled by the PMC/committers.

How will the triaging occur? (Weekly meeting?)  There will need to be some form of priority resolution and continuous feedback, lest the system run unbounded.

Having a stable+unstable release cadence + wrangling, might also help to drive community processes.

> It's not necessarily that these issues should be in the next release, or
> finished in the next quarter, but things that we'd like to encourage/guide
> contributors towards. Moreover, these are things that PMC/committers are
> prepared (and excited) to shepherd.
> 
> This doesn't mean that people can't work on other stuff, this will happen
> no matter what in OSS. More, it's to help a contributor who is trying to be
> thoughtful about what to work on to pick things from this label as they're
> more likely to get a better reception in the community. In the past we've
> had contributions that have stagnated for months because none of the
> committers have had the time to properly shepherd the contributions. 

I can't stress enough how important it is to streamline and automate our review-process (to be ruthless with time, yet gracious with people).  For example, iterating on where a bracket, or a space lies, can be maddening when a review has languished.  Don't get me wrong, I'm all for "style" & reviews, as long as they don't become an impediment to logic or sound design.   

> I'm
> not claiming we can solve that problem with JIRA labels, but I'm trying to
> push us to encourage people to work on things from our culled list to avoid
> these problems all together.
> 
> Think of it like the process for Google Summer of Code (GSOC) and Open
> Program for Women (OPW), both projects we're participating in this summer.
> We didn't just say: hey come check out our JIRA queue. Instead, we culled
> the things that we thought would be great contributions based on what was
> high-priority, what didn't have too many (if any) dependencies, etc.
> 
> Now 'active': while we can easily just look at all issues that are "In
> Progress", this can often have very poor signal to noise ratio. I want
> people to be able to look at an overview of what contributions are being
> made that have a bigger impact/influence on the project. This can provide a
> "roadmap" for the project that people from the community can easily go look
> at; it's a look into what will be done "now". So what does "now" mean?
> Ideally it's things people are working on within a few months, at most a
> few releases out (or if it's a really big impact/influence contribution
> more that a few releases out). I'm not convinced that introducing a time
> bound on these issues is really the right thing to do for the project just
> yet, and even if it is I think we can do this after we get more processes
> around labeling of issues and shepherding.
> 
> Naming: I'd love to hear recommendations from people in lieu of 'desired'.
> I'll propose 'backlog' as an alternative.
> 
> 
> Okay, so a few other questions that had come up:
> 
> * How should one prioritize outstanding reviews?
> 
> First, the expectation will be that the contributor (reviewee) needs to
> find a shepherd. At that point in time the shepherd is the first line of
> defense (for themselves!) to decide whether or not they'll be able to
> prioritize the reviews! No more available shepherds is an early indication
> that this review would likely have gone stagnant.
> 
> Of course, I don't think that *all* reviews need shepherds. Small patches
> are expected and can often get reviewed and committed very quickly. I don't
> see these as being the problem or being bottlenecked.
> 
> * Can a review/issue have multiple shepherds?
> 
> Definitely, but the shepherds need to clearly communicate that with their
> reviewee and work together to not overload the reviewee. The "pair
> shepherding" approach I mentioned in my proposal will be a case where
> multiple shepherds will be present.
> 
> 
> Alright. I'll let this sit for a bit and focus next on what everybody seems
> excited about: the shepherding process. We've got a bunch of stuff
> happening concurrently and I'd like us to start getting shepherding in
> place. Look forward to another email coming from me about that ASAP. ;-)
> 
> Thanks everyone!
> 
> Ben.
> 
> 
> 
> 
> > We need to more formally define what "active" and "desired" mean. As
> > Dominic alluded to every JIRA ticket that is *open* is desired. So it would
> > be nice to have a notion of time frame or a release that is tied with the
> > *desired* label. Probably *desired* is the wrong label to use here. This
> > helps me (and other shepherds) prioritize the reviews. Also, regarding
> > *active*, are these tickets that are assigned to someone and they have
> > started work (thinking, design, code) on it? If yes, I agree with Dominic,
> > why not just use "In Progress"?
> 
> 
> > On a related note, how do these labels tie with "Fix Version"?  Can we just
> > use "Fix version" for things that are *desired* in our next release
> > (assuming we tie *desired* to releases without explicitly tying them to a
> > time frame)?
> >
> > Another thing that is not clear is is the relationship between shepherds
> > and reviewers. Can a ticket/review have multiple shepherds? Multiple
> > reviewers? I'm assuming single shepherd and multiple reviewers to avoid
> > giving conflicting directions to contributors? Does a review need to have a
> > "ShipIt" before it can get committed?
> >
> > @Dominic: Regarding specialized committers, I'm assuming you are alluding
> > to something like having *OWNERS* files for sub components? While I think
> > it is a great idea, I don't think the project is there yet in terms of the
> > knowledge split. I would suggest following Ben's suggestion on people just
> > pinging the dev list (or IRC) to pick shepherds. If it doesn't scale or
> > creates too much noise we could rethink the approach.
> >
> > As an aside, whatever we do we should prioritize fixing tests! We already
> > add a component "test" for flaky tests but we hardly prioritize them. I
> > would love for contributors and committers to pitch in fixing the tests as
> > top priority because they are annoying and give a bad user experience.
> >
> >
> >
> > On Mon, Apr 21, 2014 at 12:27 PM, Jie Yu <yu...@gmail.com> wrote:
> >
> > > +1 for all three proposals
> > >
> > >
> > > On Mon, Apr 21, 2014 at 11:51 AM, Benjamin Hindman <
> > > benjamin.hindman@gmail.com> wrote:
> > >
> > > > The good news is, the project continues to grow! The bad news is, not
> > all
> > > > of our procedures scale. I'd like to propose some changes to streamline
> > > > hotspots around the project.
> > > >
> > > >
> > > > *(1) Review Shepherds*
> > > >
> > > > Companies that rely on Mesos expect it to be the foundation of their
> > > > software infrastructure and it's imperative that we ship high-quality
> > and
> > > > robust code. To help facilitate this we put our code through rigorous
> > > > reviews. Unfortunately, this can often act as a bottleneck, especially
> > > when
> > > > nobody wants (or has time) to review your code!
> > > >
> > > > I'd like us to be more accountable; I'd like to propose that all
> > > > significant code changes get shepherded by a PMC/committer as early in
> > > the
> > > > development phase as possible. We've played around with "review
> > > shepherds"
> > > > in the past and IMHO it's helped tremendously (and the earlier the
> > > > shepherding the better).
> > > >
> > > > Here's how I'm envisioning this would look:
> > > >
> > > > A contributor (or committer) would tell people they're interested in
> > > > working on a particular JIRA issue, feature, bug fix, TODO in the code,
> > > > etc. by either emailing dev@mesos.apache.org or posting a comment on
> > > JIRA.
> > > > Their note would specifically seek out a PMC/committer to act as a
> > > > shepherd. Note that the goal here is really to find a shepherd _before_
> > > you
> > > > start architecting or coding!
> > > >
> > > > It's possible that nobody will volunteer as a shepherd either because
> > (a)
> > > > nobody has time due to prioritizing other things in the project or (b)
> > > > they're a new PMC/committer and don't yet feel comfortable shepherding.
> > > > Seeking a shepherd early is exactly meant to deal with issues around
> > (a)
> > > > which I'll discuss in more detail below. In the case of (b), I'd like
> > to
> > > > propose people "pair" shepherd. That is, a newer PMC/committer actively
> > > > find an older (in project years) PMC/committer.
> > > >
> > > > To be clear, I don't think that all code changes should require a
> > > shepherd,
> > > > only "significant" ones. For now, I'd prefer we error on the side of
> > > > caution and seek out shepherds for most things, letting the shepherd
> > > decide
> > > > whether or not they believe the work requires them. In addition, I
> > think
> > > we
> > > > should leave it up to the shepherd's best judgement to decide when
> > design
> > > > documents or greater consensus around a certain change should be
> > sought.
> > > >
> > > > *How can you help!? *In order for this to work we'll need to actively
> > > guide
> > > > people towards finding a shepherd. Moreover, we'll need to set good
> > > > examples ourselves. People often snoop on project mailing lists and
> > mimic
> > > > the behavior of those they observe.
> > > >
> > > >
> > > > *(2) Active/Desired*
> > > >
> > > > One of the biggest reasons reviews go stagnant is because people just
> > > don't
> > > > have time to help review them. Often times this is an artifact of
> > people
> > > > picking features to work on that are low in the priority list of the
> > > > project. To *help guide people* towards issues that are *desired* I'd
> > > like
> > > > to propose that we add a new JIRA labels called, drum roll please:
> > > desired.
> > > > In conjunction with the 'desired' label I'd like to propose we also add
> > > an
> > > > 'active' label.
> > > >
> > > > An active JIRA issue is something that a
> > > contributor/committer/organization
> > > > is actively working on (or has publicly allocated time to work on in
> > the
> > > > next quarter). A desired JIRA issue is something that the
> > > > committers/organizations of the project would be working on if they had
> > > > more time! That is, things that the project community believes is of
> > > value
> > > > to the project and should get worked on.
> > > >
> > > > An advantage of labeling issues this way is that it makes creating a
> > > > "dashboard" for the project relatively easy. In fact, Chris Lambert has
> > > > already prepared one here:
> > > >
> > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=33(note:
> > > > that dashboard includes the 'newbie' label because these are also
> > > "desired"
> > > > issues just of smaller scope meant for new contributors). This
> > dashboard
> > > > can help act as basis for a roadmap for the project as well.
> > > >
> > > > To help triage what issues should be made desired I'd like to suggest
> > we
> > > > start (a) voting on tickets and recommending others vote on tickets and
> > > (b)
> > > > encourage people to make desired things known by emailing the dev@ and
> > > > user@mailing lists. In the short term I'd like to help facilitate the
> > > > triaging
> > > > via emails to the list where I can gather feedback and label tickets as
> > > > appropriate. In the long term I'd love to evolve this into
> > > > monthly/bi-weekly community meetings where people have a chance to
> > curate
> > > > desired issues.
> > > >
> > > >
> > > > *(3) Becoming a PMC Member / Committer*
> > > >
> > > > Just like code, I'd like us to be accountable for growing the
> > > PMC/committer
> > > > base. Ultimately this will give us even more shepherds enabling the
> > > project
> > > > to handle even more concurrent changes. To do this effectively I'd like
> > > to
> > > > propose that we introduce shepherds for helping contributors become
> > > > committers (sorry for the overloaded user of the word shepherd!). Like
> > > code
> > > > changes, I think we need to be more proactive about assigning a
> > shepherd
> > > to
> > > > someone that is interested in becoming a PMC/committer on the project.
> > > This
> > > > shepherd can help identify things that the contributor should
> > demonstrate
> > > > in order to be a successful PMC/committer after potentially soliciting
> > > > feedback from the existing PMC. My hope is that this will make the
> > actual
> > > > PMC/committer vote more of a formality than anything else.
> > > >
> > > >
> > > > In summary, I'd like to propose:
> > > >
> > > > Code/review shepherds.
> > > > The addition of 'active' and 'desired' JIRA labels.
> > > > PMC/committer shepherds.
> > > >
> > > > I'm clearly a +1 for these and I'm looking forward to hearing from
> > > others.
> > > >
> > > > Ben.
> > > >
> > >
> >
> 

-- 
Cheers,
Tim
Freedom, Features, Friends, First -> Fedora
https://fedoraproject.org/wiki/SIGs/bigdata

Re: scaling proposals

[Benjamin Hindman <be...@eecs.berkeley.edu>]

First, please accept my apologies for not circling back to this sooner.
It's a high-priority issue for me so I'm glad I've gotten such thoughtful
responses from everyone!

Next, let me more formally define what I mean by 'active' and 'desired'.
Before I do that, however, I want to acknowledge that they probably aren't
the best names, but let's return to that after elaborating on their purpose.

I'll start with 'desired': it is true that every JIRA ticket is indeed
"desired". Those that are not should really be resolved/closed. But not all
JIRA tickets are equal. In particular, some are more important than others,
some have dependencies that must be done before they can be started, etc. A
lot of this can be captured with existing JIRA mechanisms like priority and
dependencies. But this can quickly get inconsistent unless we triage new
tickets and monitor existing tickets very very closely.

Rather than require this constant triaging I wanted us to do more
"periodic" triaging (or on-demand when we run out of "desired" issues) and
capture the result of that via explicit labels in JIRA. That is, people
know that issues with this label have been culled by the PMC/committers.
It's not necessarily that these issues should be in the next release, or
finished in the next quarter, but things that we'd like to encourage/guide
contributors towards. Moreover, these are things that PMC/committers are
prepared (and excited) to shepherd.

This doesn't mean that people can't work on other stuff, this will happen
no matter what in OSS. More, it's to help a contributor who is trying to be
thoughtful about what to work on to pick things from this label as they're
more likely to get a better reception in the community. In the past we've
had contributions that have stagnated for months because none of the
committers have had the time to properly shepherd the contributions. I'm
not claiming we can solve that problem with JIRA labels, but I'm trying to
push us to encourage people to work on things from our culled list to avoid
these problems all together.

Think of it like the process for Google Summer of Code (GSOC) and Open
Program for Women (OPW), both projects we're participating in this summer.
We didn't just say: hey come check out our JIRA queue. Instead, we culled
the things that we thought would be great contributions based on what was
high-priority, what didn't have too many (if any) dependencies, etc.

Now 'active': while we can easily just look at all issues that are "In
Progress", this can often have very poor signal to noise ratio. I want
people to be able to look at an overview of what contributions are being
made that have a bigger impact/influence on the project. This can provide a
"roadmap" for the project that people from the community can easily go look
at; it's a look into what will be done "now". So what does "now" mean?
Ideally it's things people are working on within a few months, at most a
few releases out (or if it's a really big impact/influence contribution
more that a few releases out). I'm not convinced that introducing a time
bound on these issues is really the right thing to do for the project just
yet, and even if it is I think we can do this after we get more processes
around labeling of issues and shepherding.

Naming: I'd love to hear recommendations from people in lieu of 'desired'.
I'll propose 'backlog' as an alternative.


Okay, so a few other questions that had come up:

* How should one prioritize outstanding reviews?

First, the expectation will be that the contributor (reviewee) needs to
find a shepherd. At that point in time the shepherd is the first line of
defense (for themselves!) to decide whether or not they'll be able to
prioritize the reviews! No more available shepherds is an early indication
that this review would likely have gone stagnant.

Of course, I don't think that *all* reviews need shepherds. Small patches
are expected and can often get reviewed and committed very quickly. I don't
see these as being the problem or being bottlenecked.

* Can a review/issue have multiple shepherds?

Definitely, but the shepherds need to clearly communicate that with their
reviewee and work together to not overload the reviewee. The "pair
shepherding" approach I mentioned in my proposal will be a case where
multiple shepherds will be present.


Alright. I'll let this sit for a bit and focus next on what everybody seems
excited about: the shepherding process. We've got a bunch of stuff
happening concurrently and I'd like us to start getting shepherding in
place. Look forward to another email coming from me about that ASAP. ;-)

Thanks everyone!

Ben.




> We need to more formally define what "active" and "desired" mean. As
> Dominic alluded to every JIRA ticket that is *open* is desired. So it would
> be nice to have a notion of time frame or a release that is tied with the
> *desired* label. Probably *desired* is the wrong label to use here. This
> helps me (and other shepherds) prioritize the reviews. Also, regarding
> *active*, are these tickets that are assigned to someone and they have
> started work (thinking, design, code) on it? If yes, I agree with Dominic,
> why not just use "In Progress"?


> On a related note, how do these labels tie with "Fix Version"?  Can we just
> use "Fix version" for things that are *desired* in our next release
> (assuming we tie *desired* to releases without explicitly tying them to a
> time frame)?
>
> Another thing that is not clear is is the relationship between shepherds
> and reviewers. Can a ticket/review have multiple shepherds? Multiple
> reviewers? I'm assuming single shepherd and multiple reviewers to avoid
> giving conflicting directions to contributors? Does a review need to have a
> "ShipIt" before it can get committed?
>
> @Dominic: Regarding specialized committers, I'm assuming you are alluding
> to something like having *OWNERS* files for sub components? While I think
> it is a great idea, I don't think the project is there yet in terms of the
> knowledge split. I would suggest following Ben's suggestion on people just
> pinging the dev list (or IRC) to pick shepherds. If it doesn't scale or
> creates too much noise we could rethink the approach.
>
> As an aside, whatever we do we should prioritize fixing tests! We already
> add a component "test" for flaky tests but we hardly prioritize them. I
> would love for contributors and committers to pitch in fixing the tests as
> top priority because they are annoying and give a bad user experience.
>
>
>
> On Mon, Apr 21, 2014 at 12:27 PM, Jie Yu <yu...@gmail.com> wrote:
>
> > +1 for all three proposals
> >
> >
> > On Mon, Apr 21, 2014 at 11:51 AM, Benjamin Hindman <
> > benjamin.hindman@gmail.com> wrote:
> >
> > > The good news is, the project continues to grow! The bad news is, not
> all
> > > of our procedures scale. I'd like to propose some changes to streamline
> > > hotspots around the project.
> > >
> > >
> > > *(1) Review Shepherds*
> > >
> > > Companies that rely on Mesos expect it to be the foundation of their
> > > software infrastructure and it's imperative that we ship high-quality
> and
> > > robust code. To help facilitate this we put our code through rigorous
> > > reviews. Unfortunately, this can often act as a bottleneck, especially
> > when
> > > nobody wants (or has time) to review your code!
> > >
> > > I'd like us to be more accountable; I'd like to propose that all
> > > significant code changes get shepherded by a PMC/committer as early in
> > the
> > > development phase as possible. We've played around with "review
> > shepherds"
> > > in the past and IMHO it's helped tremendously (and the earlier the
> > > shepherding the better).
> > >
> > > Here's how I'm envisioning this would look:
> > >
> > > A contributor (or committer) would tell people they're interested in
> > > working on a particular JIRA issue, feature, bug fix, TODO in the code,
> > > etc. by either emailing dev@mesos.apache.org or posting a comment on
> > JIRA.
> > > Their note would specifically seek out a PMC/committer to act as a
> > > shepherd. Note that the goal here is really to find a shepherd _before_
> > you
> > > start architecting or coding!
> > >
> > > It's possible that nobody will volunteer as a shepherd either because
> (a)
> > > nobody has time due to prioritizing other things in the project or (b)
> > > they're a new PMC/committer and don't yet feel comfortable shepherding.
> > > Seeking a shepherd early is exactly meant to deal with issues around
> (a)
> > > which I'll discuss in more detail below. In the case of (b), I'd like
> to
> > > propose people "pair" shepherd. That is, a newer PMC/committer actively
> > > find an older (in project years) PMC/committer.
> > >
> > > To be clear, I don't think that all code changes should require a
> > shepherd,
> > > only "significant" ones. For now, I'd prefer we error on the side of
> > > caution and seek out shepherds for most things, letting the shepherd
> > decide
> > > whether or not they believe the work requires them. In addition, I
> think
> > we
> > > should leave it up to the shepherd's best judgement to decide when
> design
> > > documents or greater consensus around a certain change should be
> sought.
> > >
> > > *How can you help!? *In order for this to work we'll need to actively
> > guide
> > > people towards finding a shepherd. Moreover, we'll need to set good
> > > examples ourselves. People often snoop on project mailing lists and
> mimic
> > > the behavior of those they observe.
> > >
> > >
> > > *(2) Active/Desired*
> > >
> > > One of the biggest reasons reviews go stagnant is because people just
> > don't
> > > have time to help review them. Often times this is an artifact of
> people
> > > picking features to work on that are low in the priority list of the
> > > project. To *help guide people* towards issues that are *desired* I'd
> > like
> > > to propose that we add a new JIRA labels called, drum roll please:
> > desired.
> > > In conjunction with the 'desired' label I'd like to propose we also add
> > an
> > > 'active' label.
> > >
> > > An active JIRA issue is something that a
> > contributor/committer/organization
> > > is actively working on (or has publicly allocated time to work on in
> the
> > > next quarter). A desired JIRA issue is something that the
> > > committers/organizations of the project would be working on if they had
> > > more time! That is, things that the project community believes is of
> > value
> > > to the project and should get worked on.
> > >
> > > An advantage of labeling issues this way is that it makes creating a
> > > "dashboard" for the project relatively easy. In fact, Chris Lambert has
> > > already prepared one here:
> > >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=33(note:
> > > that dashboard includes the 'newbie' label because these are also
> > "desired"
> > > issues just of smaller scope meant for new contributors). This
> dashboard
> > > can help act as basis for a roadmap for the project as well.
> > >
> > > To help triage what issues should be made desired I'd like to suggest
> we
> > > start (a) voting on tickets and recommending others vote on tickets and
> > (b)
> > > encourage people to make desired things known by emailing the dev@ and
> > > user@mailing lists. In the short term I'd like to help facilitate the
> > > triaging
> > > via emails to the list where I can gather feedback and label tickets as
> > > appropriate. In the long term I'd love to evolve this into
> > > monthly/bi-weekly community meetings where people have a chance to
> curate
> > > desired issues.
> > >
> > >
> > > *(3) Becoming a PMC Member / Committer*
> > >
> > > Just like code, I'd like us to be accountable for growing the
> > PMC/committer
> > > base. Ultimately this will give us even more shepherds enabling the
> > project
> > > to handle even more concurrent changes. To do this effectively I'd like
> > to
> > > propose that we introduce shepherds for helping contributors become
> > > committers (sorry for the overloaded user of the word shepherd!). Like
> > code
> > > changes, I think we need to be more proactive about assigning a
> shepherd
> > to
> > > someone that is interested in becoming a PMC/committer on the project.
> > This
> > > shepherd can help identify things that the contributor should
> demonstrate
> > > in order to be a successful PMC/committer after potentially soliciting
> > > feedback from the existing PMC. My hope is that this will make the
> actual
> > > PMC/committer vote more of a formality than anything else.
> > >
> > >
> > > In summary, I'd like to propose:
> > >
> > > Code/review shepherds.
> > > The addition of 'active' and 'desired' JIRA labels.
> > > PMC/committer shepherds.
> > >
> > > I'm clearly a +1 for these and I'm looking forward to hearing from
> > others.
> > >
> > > Ben.
> > >
> >
>

Re: scaling proposals

[Brenden Matthews <br...@airbedandbreakfast.com>]

+1

I also second what Tim said.

It would be good to record these things into a wiki somewhere, for ease of
editing & amendment.

On Wed, Apr 23, 2014 at 6:22 PM, Tim St Clair <ts...@redhat.com> wrote:

> Typically the process(es) & policies that work best, can be drawn on a
> simple 'old school' flow diagram.
>
> Why? Because it becomes obvious when a process is obtuse, or ambiguous.
>
> Cheers,
> Tim
>
> ----- Original Message -----
> > From: "Vinod Kone" <vi...@gmail.com>
> > To: "dev" <de...@mesos.apache.org>
> > Sent: Monday, April 21, 2014 3:00:41 PM
> > Subject: Re: scaling proposals
> >
> > Thanks for outlining the process Ben. I have a few comments/suggestions.
> >
> > We need to more formally define what "active" and "desired" mean. As
> > Dominic alluded to every JIRA ticket that is *open* is desired. So it
> would
> > be nice to have a notion of time frame or a release that is tied with the
> > *desired* label. Probably *desired* is the wrong label to use here. This
> > helps me (and other shepherds) prioritize the reviews. Also, regarding
> > *active*, are these tickets that are assigned to someone and they have
> > started work (thinking, design, code) on it? If yes, I agree with
> Dominic,
> > why not just use "In Progress"?
> >
> > On a related note, how do these labels tie with "Fix Version"?  Can we
> just
> > use "Fix version" for things that are *desired* in our next release
> > (assuming we tie *desired* to releases without explicitly tying them to a
> > time frame)?
> >
> > Another thing that is not clear is is the relationship between shepherds
> > and reviewers. Can a ticket/review have multiple shepherds? Multiple
> > reviewers? I'm assuming single shepherd and multiple reviewers to avoid
> > giving conflicting directions to contributors? Does a review need to
> have a
> > "ShipIt" before it can get committed?
> >
> > @Dominic: Regarding specialized committers, I'm assuming you are alluding
> > to something like having *OWNERS* files for sub components? While I think
> > it is a great idea, I don't think the project is there yet in terms of
> the
> > knowledge split. I would suggest following Ben's suggestion on people
> just
> > pinging the dev list (or IRC) to pick shepherds. If it doesn't scale or
> > creates too much noise we could rethink the approach.
> >
> > As an aside, whatever we do we should prioritize fixing tests! We already
> > add a component "test" for flaky tests but we hardly prioritize them. I
> > would love for contributors and committers to pitch in fixing the tests
> as
> > top priority because they are annoying and give a bad user experience.
> >
> >
> >
> > On Mon, Apr 21, 2014 at 12:27 PM, Jie Yu <yu...@gmail.com> wrote:
> >
> > > +1 for all three proposals
> > >
> > >
> > > On Mon, Apr 21, 2014 at 11:51 AM, Benjamin Hindman <
> > > benjamin.hindman@gmail.com> wrote:
> > >
> > > > The good news is, the project continues to grow! The bad news is,
> not all
> > > > of our procedures scale. I'd like to propose some changes to
> streamline
> > > > hotspots around the project.
> > > >
> > > >
> > > > *(1) Review Shepherds*
> > > >
> > > > Companies that rely on Mesos expect it to be the foundation of their
> > > > software infrastructure and it's imperative that we ship
> high-quality and
> > > > robust code. To help facilitate this we put our code through rigorous
> > > > reviews. Unfortunately, this can often act as a bottleneck,
> especially
> > > when
> > > > nobody wants (or has time) to review your code!
> > > >
> > > > I'd like us to be more accountable; I'd like to propose that all
> > > > significant code changes get shepherded by a PMC/committer as early
> in
> > > the
> > > > development phase as possible. We've played around with "review
> > > shepherds"
> > > > in the past and IMHO it's helped tremendously (and the earlier the
> > > > shepherding the better).
> > > >
> > > > Here's how I'm envisioning this would look:
> > > >
> > > > A contributor (or committer) would tell people they're interested in
> > > > working on a particular JIRA issue, feature, bug fix, TODO in the
> code,
> > > > etc. by either emailing dev@mesos.apache.org or posting a comment on
> > > JIRA.
> > > > Their note would specifically seek out a PMC/committer to act as a
> > > > shepherd. Note that the goal here is really to find a shepherd
> _before_
> > > you
> > > > start architecting or coding!
> > > >
> > > > It's possible that nobody will volunteer as a shepherd either
> because (a)
> > > > nobody has time due to prioritizing other things in the project or
> (b)
> > > > they're a new PMC/committer and don't yet feel comfortable
> shepherding.
> > > > Seeking a shepherd early is exactly meant to deal with issues around
> (a)
> > > > which I'll discuss in more detail below. In the case of (b), I'd
> like to
> > > > propose people "pair" shepherd. That is, a newer PMC/committer
> actively
> > > > find an older (in project years) PMC/committer.
> > > >
> > > > To be clear, I don't think that all code changes should require a
> > > shepherd,
> > > > only "significant" ones. For now, I'd prefer we error on the side of
> > > > caution and seek out shepherds for most things, letting the shepherd
> > > decide
> > > > whether or not they believe the work requires them. In addition, I
> think
> > > we
> > > > should leave it up to the shepherd's best judgement to decide when
> design
> > > > documents or greater consensus around a certain change should be
> sought.
> > > >
> > > > *How can you help!? *In order for this to work we'll need to actively
> > > guide
> > > > people towards finding a shepherd. Moreover, we'll need to set good
> > > > examples ourselves. People often snoop on project mailing lists and
> mimic
> > > > the behavior of those they observe.
> > > >
> > > >
> > > > *(2) Active/Desired*
> > > >
> > > > One of the biggest reasons reviews go stagnant is because people just
> > > don't
> > > > have time to help review them. Often times this is an artifact of
> people
> > > > picking features to work on that are low in the priority list of the
> > > > project. To *help guide people* towards issues that are *desired* I'd
> > > like
> > > > to propose that we add a new JIRA labels called, drum roll please:
> > > desired.
> > > > In conjunction with the 'desired' label I'd like to propose we also
> add
> > > an
> > > > 'active' label.
> > > >
> > > > An active JIRA issue is something that a
> > > contributor/committer/organization
> > > > is actively working on (or has publicly allocated time to work on in
> the
> > > > next quarter). A desired JIRA issue is something that the
> > > > committers/organizations of the project would be working on if they
> had
> > > > more time! That is, things that the project community believes is of
> > > value
> > > > to the project and should get worked on.
> > > >
> > > > An advantage of labeling issues this way is that it makes creating a
> > > > "dashboard" for the project relatively easy. In fact, Chris Lambert
> has
> > > > already prepared one here:
> > > >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=33(note:
> > > > that dashboard includes the 'newbie' label because these are also
> > > "desired"
> > > > issues just of smaller scope meant for new contributors). This
> dashboard
> > > > can help act as basis for a roadmap for the project as well.
> > > >
> > > > To help triage what issues should be made desired I'd like to
> suggest we
> > > > start (a) voting on tickets and recommending others vote on tickets
> and
> > > (b)
> > > > encourage people to make desired things known by emailing the dev@and
> > > > user@mailing lists. In the short term I'd like to help facilitate
> the
> > > > triaging
> > > > via emails to the list where I can gather feedback and label tickets
> as
> > > > appropriate. In the long term I'd love to evolve this into
> > > > monthly/bi-weekly community meetings where people have a chance to
> curate
> > > > desired issues.
> > > >
> > > >
> > > > *(3) Becoming a PMC Member / Committer*
> > > >
> > > > Just like code, I'd like us to be accountable for growing the
> > > PMC/committer
> > > > base. Ultimately this will give us even more shepherds enabling the
> > > project
> > > > to handle even more concurrent changes. To do this effectively I'd
> like
> > > to
> > > > propose that we introduce shepherds for helping contributors become
> > > > committers (sorry for the overloaded user of the word shepherd!).
> Like
> > > code
> > > > changes, I think we need to be more proactive about assigning a
> shepherd
> > > to
> > > > someone that is interested in becoming a PMC/committer on the
> project.
> > > This
> > > > shepherd can help identify things that the contributor should
> demonstrate
> > > > in order to be a successful PMC/committer after potentially
> soliciting
> > > > feedback from the existing PMC. My hope is that this will make the
> actual
> > > > PMC/committer vote more of a formality than anything else.
> > > >
> > > >
> > > > In summary, I'd like to propose:
> > > >
> > > > Code/review shepherds.
> > > > The addition of 'active' and 'desired' JIRA labels.
> > > > PMC/committer shepherds.
> > > >
> > > > I'm clearly a +1 for these and I'm looking forward to hearing from
> > > others.
> > > >
> > > > Ben.
> > > >
> > >
> >
>
> --
> Cheers,
> Tim
> Freedom, Features, Friends, First -> Fedora
> https://fedoraproject.org/wiki/SIGs/bigdata
>

Re: scaling proposals

[Tim St Clair <ts...@redhat.com>]

Typically the process(es) & policies that work best, can be drawn on a simple 'old school' flow diagram. 

Why? Because it becomes obvious when a process is obtuse, or ambiguous.    

Cheers,
Tim

----- Original Message -----
> From: "Vinod Kone" <vi...@gmail.com>
> To: "dev" <de...@mesos.apache.org>
> Sent: Monday, April 21, 2014 3:00:41 PM
> Subject: Re: scaling proposals
> 
> Thanks for outlining the process Ben. I have a few comments/suggestions.
> 
> We need to more formally define what "active" and "desired" mean. As
> Dominic alluded to every JIRA ticket that is *open* is desired. So it would
> be nice to have a notion of time frame or a release that is tied with the
> *desired* label. Probably *desired* is the wrong label to use here. This
> helps me (and other shepherds) prioritize the reviews. Also, regarding
> *active*, are these tickets that are assigned to someone and they have
> started work (thinking, design, code) on it? If yes, I agree with Dominic,
> why not just use "In Progress"?
> 
> On a related note, how do these labels tie with "Fix Version"?  Can we just
> use "Fix version" for things that are *desired* in our next release
> (assuming we tie *desired* to releases without explicitly tying them to a
> time frame)?
> 
> Another thing that is not clear is is the relationship between shepherds
> and reviewers. Can a ticket/review have multiple shepherds? Multiple
> reviewers? I'm assuming single shepherd and multiple reviewers to avoid
> giving conflicting directions to contributors? Does a review need to have a
> "ShipIt" before it can get committed?
> 
> @Dominic: Regarding specialized committers, I'm assuming you are alluding
> to something like having *OWNERS* files for sub components? While I think
> it is a great idea, I don't think the project is there yet in terms of the
> knowledge split. I would suggest following Ben's suggestion on people just
> pinging the dev list (or IRC) to pick shepherds. If it doesn't scale or
> creates too much noise we could rethink the approach.
> 
> As an aside, whatever we do we should prioritize fixing tests! We already
> add a component "test" for flaky tests but we hardly prioritize them. I
> would love for contributors and committers to pitch in fixing the tests as
> top priority because they are annoying and give a bad user experience.
> 
> 
> 
> On Mon, Apr 21, 2014 at 12:27 PM, Jie Yu <yu...@gmail.com> wrote:
> 
> > +1 for all three proposals
> >
> >
> > On Mon, Apr 21, 2014 at 11:51 AM, Benjamin Hindman <
> > benjamin.hindman@gmail.com> wrote:
> >
> > > The good news is, the project continues to grow! The bad news is, not all
> > > of our procedures scale. I'd like to propose some changes to streamline
> > > hotspots around the project.
> > >
> > >
> > > *(1) Review Shepherds*
> > >
> > > Companies that rely on Mesos expect it to be the foundation of their
> > > software infrastructure and it's imperative that we ship high-quality and
> > > robust code. To help facilitate this we put our code through rigorous
> > > reviews. Unfortunately, this can often act as a bottleneck, especially
> > when
> > > nobody wants (or has time) to review your code!
> > >
> > > I'd like us to be more accountable; I'd like to propose that all
> > > significant code changes get shepherded by a PMC/committer as early in
> > the
> > > development phase as possible. We've played around with "review
> > shepherds"
> > > in the past and IMHO it's helped tremendously (and the earlier the
> > > shepherding the better).
> > >
> > > Here's how I'm envisioning this would look:
> > >
> > > A contributor (or committer) would tell people they're interested in
> > > working on a particular JIRA issue, feature, bug fix, TODO in the code,
> > > etc. by either emailing dev@mesos.apache.org or posting a comment on
> > JIRA.
> > > Their note would specifically seek out a PMC/committer to act as a
> > > shepherd. Note that the goal here is really to find a shepherd _before_
> > you
> > > start architecting or coding!
> > >
> > > It's possible that nobody will volunteer as a shepherd either because (a)
> > > nobody has time due to prioritizing other things in the project or (b)
> > > they're a new PMC/committer and don't yet feel comfortable shepherding.
> > > Seeking a shepherd early is exactly meant to deal with issues around (a)
> > > which I'll discuss in more detail below. In the case of (b), I'd like to
> > > propose people "pair" shepherd. That is, a newer PMC/committer actively
> > > find an older (in project years) PMC/committer.
> > >
> > > To be clear, I don't think that all code changes should require a
> > shepherd,
> > > only "significant" ones. For now, I'd prefer we error on the side of
> > > caution and seek out shepherds for most things, letting the shepherd
> > decide
> > > whether or not they believe the work requires them. In addition, I think
> > we
> > > should leave it up to the shepherd's best judgement to decide when design
> > > documents or greater consensus around a certain change should be sought.
> > >
> > > *How can you help!? *In order for this to work we'll need to actively
> > guide
> > > people towards finding a shepherd. Moreover, we'll need to set good
> > > examples ourselves. People often snoop on project mailing lists and mimic
> > > the behavior of those they observe.
> > >
> > >
> > > *(2) Active/Desired*
> > >
> > > One of the biggest reasons reviews go stagnant is because people just
> > don't
> > > have time to help review them. Often times this is an artifact of people
> > > picking features to work on that are low in the priority list of the
> > > project. To *help guide people* towards issues that are *desired* I'd
> > like
> > > to propose that we add a new JIRA labels called, drum roll please:
> > desired.
> > > In conjunction with the 'desired' label I'd like to propose we also add
> > an
> > > 'active' label.
> > >
> > > An active JIRA issue is something that a
> > contributor/committer/organization
> > > is actively working on (or has publicly allocated time to work on in the
> > > next quarter). A desired JIRA issue is something that the
> > > committers/organizations of the project would be working on if they had
> > > more time! That is, things that the project community believes is of
> > value
> > > to the project and should get worked on.
> > >
> > > An advantage of labeling issues this way is that it makes creating a
> > > "dashboard" for the project relatively easy. In fact, Chris Lambert has
> > > already prepared one here:
> > > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=33(note:
> > > that dashboard includes the 'newbie' label because these are also
> > "desired"
> > > issues just of smaller scope meant for new contributors). This dashboard
> > > can help act as basis for a roadmap for the project as well.
> > >
> > > To help triage what issues should be made desired I'd like to suggest we
> > > start (a) voting on tickets and recommending others vote on tickets and
> > (b)
> > > encourage people to make desired things known by emailing the dev@ and
> > > user@mailing lists. In the short term I'd like to help facilitate the
> > > triaging
> > > via emails to the list where I can gather feedback and label tickets as
> > > appropriate. In the long term I'd love to evolve this into
> > > monthly/bi-weekly community meetings where people have a chance to curate
> > > desired issues.
> > >
> > >
> > > *(3) Becoming a PMC Member / Committer*
> > >
> > > Just like code, I'd like us to be accountable for growing the
> > PMC/committer
> > > base. Ultimately this will give us even more shepherds enabling the
> > project
> > > to handle even more concurrent changes. To do this effectively I'd like
> > to
> > > propose that we introduce shepherds for helping contributors become
> > > committers (sorry for the overloaded user of the word shepherd!). Like
> > code
> > > changes, I think we need to be more proactive about assigning a shepherd
> > to
> > > someone that is interested in becoming a PMC/committer on the project.
> > This
> > > shepherd can help identify things that the contributor should demonstrate
> > > in order to be a successful PMC/committer after potentially soliciting
> > > feedback from the existing PMC. My hope is that this will make the actual
> > > PMC/committer vote more of a formality than anything else.
> > >
> > >
> > > In summary, I'd like to propose:
> > >
> > > Code/review shepherds.
> > > The addition of 'active' and 'desired' JIRA labels.
> > > PMC/committer shepherds.
> > >
> > > I'm clearly a +1 for these and I'm looking forward to hearing from
> > others.
> > >
> > > Ben.
> > >
> >
> 

-- 
Cheers,
Tim
Freedom, Features, Friends, First -> Fedora
https://fedoraproject.org/wiki/SIGs/bigdata

Re: scaling proposals

[Vinod Kone <vi...@gmail.com>]

Thanks for outlining the process Ben. I have a few comments/suggestions.

We need to more formally define what "active" and "desired" mean. As
Dominic alluded to every JIRA ticket that is *open* is desired. So it would
be nice to have a notion of time frame or a release that is tied with the
*desired* label. Probably *desired* is the wrong label to use here. This
helps me (and other shepherds) prioritize the reviews. Also, regarding
*active*, are these tickets that are assigned to someone and they have
started work (thinking, design, code) on it? If yes, I agree with Dominic,
why not just use "In Progress"?

On a related note, how do these labels tie with "Fix Version"?  Can we just
use "Fix version" for things that are *desired* in our next release
(assuming we tie *desired* to releases without explicitly tying them to a
time frame)?

Another thing that is not clear is is the relationship between shepherds
and reviewers. Can a ticket/review have multiple shepherds? Multiple
reviewers? I'm assuming single shepherd and multiple reviewers to avoid
giving conflicting directions to contributors? Does a review need to have a
"ShipIt" before it can get committed?

@Dominic: Regarding specialized committers, I'm assuming you are alluding
to something like having *OWNERS* files for sub components? While I think
it is a great idea, I don't think the project is there yet in terms of the
knowledge split. I would suggest following Ben's suggestion on people just
pinging the dev list (or IRC) to pick shepherds. If it doesn't scale or
creates too much noise we could rethink the approach.

As an aside, whatever we do we should prioritize fixing tests! We already
add a component "test" for flaky tests but we hardly prioritize them. I
would love for contributors and committers to pitch in fixing the tests as
top priority because they are annoying and give a bad user experience.



On Mon, Apr 21, 2014 at 12:27 PM, Jie Yu <yu...@gmail.com> wrote:

> +1 for all three proposals
>
>
> On Mon, Apr 21, 2014 at 11:51 AM, Benjamin Hindman <
> benjamin.hindman@gmail.com> wrote:
>
> > The good news is, the project continues to grow! The bad news is, not all
> > of our procedures scale. I'd like to propose some changes to streamline
> > hotspots around the project.
> >
> >
> > *(1) Review Shepherds*
> >
> > Companies that rely on Mesos expect it to be the foundation of their
> > software infrastructure and it's imperative that we ship high-quality and
> > robust code. To help facilitate this we put our code through rigorous
> > reviews. Unfortunately, this can often act as a bottleneck, especially
> when
> > nobody wants (or has time) to review your code!
> >
> > I'd like us to be more accountable; I'd like to propose that all
> > significant code changes get shepherded by a PMC/committer as early in
> the
> > development phase as possible. We've played around with "review
> shepherds"
> > in the past and IMHO it's helped tremendously (and the earlier the
> > shepherding the better).
> >
> > Here's how I'm envisioning this would look:
> >
> > A contributor (or committer) would tell people they're interested in
> > working on a particular JIRA issue, feature, bug fix, TODO in the code,
> > etc. by either emailing dev@mesos.apache.org or posting a comment on
> JIRA.
> > Their note would specifically seek out a PMC/committer to act as a
> > shepherd. Note that the goal here is really to find a shepherd _before_
> you
> > start architecting or coding!
> >
> > It's possible that nobody will volunteer as a shepherd either because (a)
> > nobody has time due to prioritizing other things in the project or (b)
> > they're a new PMC/committer and don't yet feel comfortable shepherding.
> > Seeking a shepherd early is exactly meant to deal with issues around (a)
> > which I'll discuss in more detail below. In the case of (b), I'd like to
> > propose people "pair" shepherd. That is, a newer PMC/committer actively
> > find an older (in project years) PMC/committer.
> >
> > To be clear, I don't think that all code changes should require a
> shepherd,
> > only "significant" ones. For now, I'd prefer we error on the side of
> > caution and seek out shepherds for most things, letting the shepherd
> decide
> > whether or not they believe the work requires them. In addition, I think
> we
> > should leave it up to the shepherd's best judgement to decide when design
> > documents or greater consensus around a certain change should be sought.
> >
> > *How can you help!? *In order for this to work we'll need to actively
> guide
> > people towards finding a shepherd. Moreover, we'll need to set good
> > examples ourselves. People often snoop on project mailing lists and mimic
> > the behavior of those they observe.
> >
> >
> > *(2) Active/Desired*
> >
> > One of the biggest reasons reviews go stagnant is because people just
> don't
> > have time to help review them. Often times this is an artifact of people
> > picking features to work on that are low in the priority list of the
> > project. To *help guide people* towards issues that are *desired* I'd
> like
> > to propose that we add a new JIRA labels called, drum roll please:
> desired.
> > In conjunction with the 'desired' label I'd like to propose we also add
> an
> > 'active' label.
> >
> > An active JIRA issue is something that a
> contributor/committer/organization
> > is actively working on (or has publicly allocated time to work on in the
> > next quarter). A desired JIRA issue is something that the
> > committers/organizations of the project would be working on if they had
> > more time! That is, things that the project community believes is of
> value
> > to the project and should get worked on.
> >
> > An advantage of labeling issues this way is that it makes creating a
> > "dashboard" for the project relatively easy. In fact, Chris Lambert has
> > already prepared one here:
> > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=33(note:
> > that dashboard includes the 'newbie' label because these are also
> "desired"
> > issues just of smaller scope meant for new contributors). This dashboard
> > can help act as basis for a roadmap for the project as well.
> >
> > To help triage what issues should be made desired I'd like to suggest we
> > start (a) voting on tickets and recommending others vote on tickets and
> (b)
> > encourage people to make desired things known by emailing the dev@ and
> > user@mailing lists. In the short term I'd like to help facilitate the
> > triaging
> > via emails to the list where I can gather feedback and label tickets as
> > appropriate. In the long term I'd love to evolve this into
> > monthly/bi-weekly community meetings where people have a chance to curate
> > desired issues.
> >
> >
> > *(3) Becoming a PMC Member / Committer*
> >
> > Just like code, I'd like us to be accountable for growing the
> PMC/committer
> > base. Ultimately this will give us even more shepherds enabling the
> project
> > to handle even more concurrent changes. To do this effectively I'd like
> to
> > propose that we introduce shepherds for helping contributors become
> > committers (sorry for the overloaded user of the word shepherd!). Like
> code
> > changes, I think we need to be more proactive about assigning a shepherd
> to
> > someone that is interested in becoming a PMC/committer on the project.
> This
> > shepherd can help identify things that the contributor should demonstrate
> > in order to be a successful PMC/committer after potentially soliciting
> > feedback from the existing PMC. My hope is that this will make the actual
> > PMC/committer vote more of a formality than anything else.
> >
> >
> > In summary, I'd like to propose:
> >
> > Code/review shepherds.
> > The addition of 'active' and 'desired' JIRA labels.
> > PMC/committer shepherds.
> >
> > I'm clearly a +1 for these and I'm looking forward to hearing from
> others.
> >
> > Ben.
> >
>

Re: scaling proposals

[Jie Yu <yu...@gmail.com>]

+1 for all three proposals


On Mon, Apr 21, 2014 at 11:51 AM, Benjamin Hindman <
benjamin.hindman@gmail.com> wrote:

> The good news is, the project continues to grow! The bad news is, not all
> of our procedures scale. I'd like to propose some changes to streamline
> hotspots around the project.
>
>
> *(1) Review Shepherds*
>
> Companies that rely on Mesos expect it to be the foundation of their
> software infrastructure and it's imperative that we ship high-quality and
> robust code. To help facilitate this we put our code through rigorous
> reviews. Unfortunately, this can often act as a bottleneck, especially when
> nobody wants (or has time) to review your code!
>
> I'd like us to be more accountable; I'd like to propose that all
> significant code changes get shepherded by a PMC/committer as early in the
> development phase as possible. We've played around with "review shepherds"
> in the past and IMHO it's helped tremendously (and the earlier the
> shepherding the better).
>
> Here's how I'm envisioning this would look:
>
> A contributor (or committer) would tell people they're interested in
> working on a particular JIRA issue, feature, bug fix, TODO in the code,
> etc. by either emailing dev@mesos.apache.org or posting a comment on JIRA.
> Their note would specifically seek out a PMC/committer to act as a
> shepherd. Note that the goal here is really to find a shepherd _before_ you
> start architecting or coding!
>
> It's possible that nobody will volunteer as a shepherd either because (a)
> nobody has time due to prioritizing other things in the project or (b)
> they're a new PMC/committer and don't yet feel comfortable shepherding.
> Seeking a shepherd early is exactly meant to deal with issues around (a)
> which I'll discuss in more detail below. In the case of (b), I'd like to
> propose people "pair" shepherd. That is, a newer PMC/committer actively
> find an older (in project years) PMC/committer.
>
> To be clear, I don't think that all code changes should require a shepherd,
> only "significant" ones. For now, I'd prefer we error on the side of
> caution and seek out shepherds for most things, letting the shepherd decide
> whether or not they believe the work requires them. In addition, I think we
> should leave it up to the shepherd's best judgement to decide when design
> documents or greater consensus around a certain change should be sought.
>
> *How can you help!? *In order for this to work we'll need to actively guide
> people towards finding a shepherd. Moreover, we'll need to set good
> examples ourselves. People often snoop on project mailing lists and mimic
> the behavior of those they observe.
>
>
> *(2) Active/Desired*
>
> One of the biggest reasons reviews go stagnant is because people just don't
> have time to help review them. Often times this is an artifact of people
> picking features to work on that are low in the priority list of the
> project. To *help guide people* towards issues that are *desired* I'd like
> to propose that we add a new JIRA labels called, drum roll please: desired.
> In conjunction with the 'desired' label I'd like to propose we also add an
> 'active' label.
>
> An active JIRA issue is something that a contributor/committer/organization
> is actively working on (or has publicly allocated time to work on in the
> next quarter). A desired JIRA issue is something that the
> committers/organizations of the project would be working on if they had
> more time! That is, things that the project community believes is of value
> to the project and should get worked on.
>
> An advantage of labeling issues this way is that it makes creating a
> "dashboard" for the project relatively easy. In fact, Chris Lambert has
> already prepared one here:
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=33 (note:
> that dashboard includes the 'newbie' label because these are also "desired"
> issues just of smaller scope meant for new contributors). This dashboard
> can help act as basis for a roadmap for the project as well.
>
> To help triage what issues should be made desired I'd like to suggest we
> start (a) voting on tickets and recommending others vote on tickets and (b)
> encourage people to make desired things known by emailing the dev@ and
> user@mailing lists. In the short term I'd like to help facilitate the
> triaging
> via emails to the list where I can gather feedback and label tickets as
> appropriate. In the long term I'd love to evolve this into
> monthly/bi-weekly community meetings where people have a chance to curate
> desired issues.
>
>
> *(3) Becoming a PMC Member / Committer*
>
> Just like code, I'd like us to be accountable for growing the PMC/committer
> base. Ultimately this will give us even more shepherds enabling the project
> to handle even more concurrent changes. To do this effectively I'd like to
> propose that we introduce shepherds for helping contributors become
> committers (sorry for the overloaded user of the word shepherd!). Like code
> changes, I think we need to be more proactive about assigning a shepherd to
> someone that is interested in becoming a PMC/committer on the project. This
> shepherd can help identify things that the contributor should demonstrate
> in order to be a successful PMC/committer after potentially soliciting
> feedback from the existing PMC. My hope is that this will make the actual
> PMC/committer vote more of a formality than anything else.
>
>
> In summary, I'd like to propose:
>
> Code/review shepherds.
> The addition of 'active' and 'desired' JIRA labels.
> PMC/committer shepherds.
>
> I'm clearly a +1 for these and I'm looking forward to hearing from others.
>
> Ben.
>

Re: scaling proposals

[Niklas Nielsen <ni...@qni.dk>]

Strong +1 - great to get some more clarity on the process!

Thanks Ben


On Mon, Apr 21, 2014 at 11:51 AM, Benjamin Hindman <
benjamin.hindman@gmail.com> wrote:

> The good news is, the project continues to grow! The bad news is, not all
> of our procedures scale. I'd like to propose some changes to streamline
> hotspots around the project.
>
>
> *(1) Review Shepherds*
>
> Companies that rely on Mesos expect it to be the foundation of their
> software infrastructure and it's imperative that we ship high-quality and
> robust code. To help facilitate this we put our code through rigorous
> reviews. Unfortunately, this can often act as a bottleneck, especially when
> nobody wants (or has time) to review your code!
>
> I'd like us to be more accountable; I'd like to propose that all
> significant code changes get shepherded by a PMC/committer as early in the
> development phase as possible. We've played around with "review shepherds"
> in the past and IMHO it's helped tremendously (and the earlier the
> shepherding the better).
>
> Here's how I'm envisioning this would look:
>
> A contributor (or committer) would tell people they're interested in
> working on a particular JIRA issue, feature, bug fix, TODO in the code,
> etc. by either emailing dev@mesos.apache.org or posting a comment on JIRA.
> Their note would specifically seek out a PMC/committer to act as a
> shepherd. Note that the goal here is really to find a shepherd _before_ you
> start architecting or coding!
>
> It's possible that nobody will volunteer as a shepherd either because (a)
> nobody has time due to prioritizing other things in the project or (b)
> they're a new PMC/committer and don't yet feel comfortable shepherding.
> Seeking a shepherd early is exactly meant to deal with issues around (a)
> which I'll discuss in more detail below. In the case of (b), I'd like to
> propose people "pair" shepherd. That is, a newer PMC/committer actively
> find an older (in project years) PMC/committer.
>
> To be clear, I don't think that all code changes should require a shepherd,
> only "significant" ones. For now, I'd prefer we error on the side of
> caution and seek out shepherds for most things, letting the shepherd decide
> whether or not they believe the work requires them. In addition, I think we
> should leave it up to the shepherd's best judgement to decide when design
> documents or greater consensus around a certain change should be sought.
>
> *How can you help!? *In order for this to work we'll need to actively guide
> people towards finding a shepherd. Moreover, we'll need to set good
> examples ourselves. People often snoop on project mailing lists and mimic
> the behavior of those they observe.
>
>
> *(2) Active/Desired*
>
> One of the biggest reasons reviews go stagnant is because people just don't
> have time to help review them. Often times this is an artifact of people
> picking features to work on that are low in the priority list of the
> project. To *help guide people* towards issues that are *desired* I'd like
> to propose that we add a new JIRA labels called, drum roll please: desired.
> In conjunction with the 'desired' label I'd like to propose we also add an
> 'active' label.
>
> An active JIRA issue is something that a contributor/committer/organization
> is actively working on (or has publicly allocated time to work on in the
> next quarter). A desired JIRA issue is something that the
> committers/organizations of the project would be working on if they had
> more time! That is, things that the project community believes is of value
> to the project and should get worked on.
>
> An advantage of labeling issues this way is that it makes creating a
> "dashboard" for the project relatively easy. In fact, Chris Lambert has
> already prepared one here:
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=33 (note:
> that dashboard includes the 'newbie' label because these are also "desired"
> issues just of smaller scope meant for new contributors). This dashboard
> can help act as basis for a roadmap for the project as well.
>
> To help triage what issues should be made desired I'd like to suggest we
> start (a) voting on tickets and recommending others vote on tickets and (b)
> encourage people to make desired things known by emailing the dev@ and
> user@mailing lists. In the short term I'd like to help facilitate the
> triaging
> via emails to the list where I can gather feedback and label tickets as
> appropriate. In the long term I'd love to evolve this into
> monthly/bi-weekly community meetings where people have a chance to curate
> desired issues.
>
>
> *(3) Becoming a PMC Member / Committer*
>
> Just like code, I'd like us to be accountable for growing the PMC/committer
> base. Ultimately this will give us even more shepherds enabling the project
> to handle even more concurrent changes. To do this effectively I'd like to
> propose that we introduce shepherds for helping contributors become
> committers (sorry for the overloaded user of the word shepherd!). Like code
> changes, I think we need to be more proactive about assigning a shepherd to
> someone that is interested in becoming a PMC/committer on the project. This
> shepherd can help identify things that the contributor should demonstrate
> in order to be a successful PMC/committer after potentially soliciting
> feedback from the existing PMC. My hope is that this will make the actual
> PMC/committer vote more of a formality than anything else.
>
>
> In summary, I'd like to propose:
>
> Code/review shepherds.
> The addition of 'active' and 'desired' JIRA labels.
> PMC/committer shepherds.
>
> I'm clearly a +1 for these and I'm looking forward to hearing from others.
>
> Ben.