Reconsidering the "buddy system"

[Max Bowsher <ma...@ukf.net>]

These days, the dev@ list is a very busy place. It takes a truly 
dedicated person to attempt to read everything that goes on there. I 
regularly find myself discarding some threads, in order to reserve 
enough time to participate effectively in others. In particular, I am 
concerned about people posting patches which slip past our collective 
radar and are lost in oblivion. For this reason, I wonder if we have 
done wrong in trying to direct patches to email and never issue-tracking.


I've recently gone and tried to read our issue tracker policy from an 
outside perspective. The result of that is that I am now afraid that we 
are making unreasonable demands on what people must do before they 
report bugs to us. I've come to the conclusion that if I was to 
encounter Subversion's policy for the first time in a project that I was 
not strongly involved in, I would be fairly indignant at the hoops I was 
being asked to jump through.


For the two reasons above, I would like to propose that the buddy policy 
be abolished forthwith.


Naturally, this will require that we developers interact more with the 
issue tracker. However, I think this is something we MUST do anyway - 
there are currently 72 issues in the "---" milestone, which is supposed 
to be a marker of "this issue has not yet received initial triage". I 
think we have fallen into the trap of marginalizing the issue tracker to 
the point that even the developers aren't using it effectively, for the 
most part.


So, in brief, here's what I think we should do about it:


1. Modify the IZ front page to:

a) politely request that people carefully consider whether they have a 
true bug, or a case of user error, or a reasonably complete code 
contribution, or a simple draft patch for discussion.

b) guide people to use the issue tracker for bugs or significant code 
contribution, or the mailing list otherwise.

c) strongly recommend that when filing an issue, they also post a link 
and a brief summary to dev@, explaining that this will greatly help in 
getting prompt responses. Emphasize that the list is the place for 
discussion, and the issue tracker works best for checkpointing summaries 
of the progress of discussions, not holding them itself.


2. Set up a simple list of boilerplate responses for closing misguided 
issue filings, so that we can deal with them with an extreme minimum of 
effort.


3. Have a policy of regularly reviewing "---"-milestoned issues, perhaps 
with an automated periodic mailing of open "---" issues to dev.


4. Try to use IZ to maintain a central database of things that need to 
be done, rather than keeping a great deal of that information on 
individual brains and inboxes.



I am hopeful that the end result of this will be to take some of the 
pressure off dev@ as the sole mechanism for reporting *and tracking* 
inbound bug reports, and allow us to make better use of our issue tracker.

Max.

Re: Reconsidering the "buddy system"

[Karl Fogel <kf...@google.com>]

Max Bowsher <ma...@ukf.net> writes:
> These days, the dev@ list is a very busy place. It takes a truly
> dedicated person to attempt to read everything that goes on there. I
> regularly find myself discarding some threads, in order to reserve
> enough time to participate effectively in others. In particular, I am
> concerned about people posting patches which slip past our collective
> radar and are lost in oblivion. For this reason, I wonder if we have
> done wrong in trying to direct patches to email and never
> issue-tracking.

This is exactly why we have a Patch Manager.  If that person isn't
keeping up -- I'm not saying he is or isn't, I've not been in touch
enough recently to know -- our community is easily large enough to
find another qualified volunteer.

The Patch Manager system works very well as long as the manager stays
on top of things.  Is that not happening here?

> I've recently gone and tried to read our issue tracker policy from an
> outside perspective. The result of that is that I am now afraid that
> we are making unreasonable demands on what people must do before they
> report bugs to us. I've come to the conclusion that if I was to
> encounter Subversion's policy for the first time in a project that I
> was not strongly involved in, I would be fairly indignant at the hoops
> I was being asked to jump through.
>
>
> For the two reasons above, I would like to propose that the buddy
> policy be abolished forthwith.

I'm not sure I agree with this analysis.

The Buddy System does not require dev@ list interaction.  IRC is fine
too.  And private (non-list) email is fine.  All the Buddy System
should really require is that you get someone else to agree with you
that this is a bug.  The wording on the issues page should be changed
to make this clear -- right now it points to users@ and dev@
specifically; we should have it state explicitly that we just want at
least two heads agreeing on each bug report.

I'll be happy to make this change, if needed, once this conversation
has resolved.

(To emphasize, the Buddy System is only about bug reports, not
patches.  I'm not sure the two should even be considered in the same
thread, they're quite different -- the main similarity is that we use
the issue tracker for tracking the lifecycles of both).

> Naturally, this will require that we developers interact more with the
> issue tracker. However, I think this is something we MUST do anyway - 
> there are currently 72 issues in the "---" milestone, which is
> supposed to be a marker of "this issue has not yet received initial
> triage". I think we have fallen into the trap of marginalizing the
> issue tracker to the point that even the developers aren't using it
> effectively, for the most part.

Why would changing the Buddy System requirement result in developers
spending more time triaging issues?  If they're not doing it now, when
the issues have already been through one level of filtering and are
presumably of high quality, then what is it about a higher number of
lower-quality issues that would cause developers to spend *more* time
in the tracker, exactly? :-)

> So, in brief, here's what I think we should do about it:
>
> 1. Modify the IZ front page to:
>
> a) politely request that people carefully consider whether they have a
> true bug, or a case of user error, or a reasonably complete code
> contribution, or a simple draft patch for discussion.
>
> b) guide people to use the issue tracker for bugs or significant code
> contribution, or the mailing list otherwise.
>
> c) strongly recommend that when filing an issue, they also post a link
> and a brief summary to dev@, explaining that this will greatly help in
> getting prompt responses. Emphasize that the list is the place for
> discussion, and the issue tracker works best for checkpointing
> summaries of the progress of discussions, not holding them itself.
>
>
> 2. Set up a simple list of boilerplate responses for closing misguided
> issue filings, so that we can deal with them with an extreme minimum
> of effort.

A lot of interface clunkiness is disguised behind that "set up a
simple list".  The fact is, dealing with any issue is always going to
take a certain constant level of effort.  I already have boilerplate
responses, sitting unused in files; maybe if my browser were more
easily scriptable I'd use them :-).

> 3. Have a policy of regularly reviewing "---"-milestoned issues,
> perhaps with an automated periodic mailing of open "---" issues to
> dev.

Policies that are "negative action" are successful here because they
specify requirements that must be met before something is done, e.g.,
a certain number of signatures on a release tarball.

But "positive action" policies are totally different.  Okay, so we
have this policy; but who actually is going to do this reviewing?
Whoever those people are, why are they more likely to do it when we
have a policy then when we don't?  Are we giving them raises or
something?

> 4. Try to use IZ to maintain a central database of things that need to
> be done, rather than keeping a great deal of that information on
> individual brains and inboxes.

I thought we already do this.  Do we not?

> I am hopeful that the end result of this will be to take some of the
> pressure off dev@ as the sole mechanism for reporting *and tracking*
> inbound bug reports, and allow us to make better use of our issue
> tracker.

dev@ is not the sole mechanism for tracking inbound bug reports.

dev@ has been the sole mechanism for receiving (but not tracking)
inbound *patches*, and a patch manager has been responsible for seeing
to it that they make it into issues when not processed immediately
on-list.  If that system has lapsed, let's find out why.  It's a good
system when it works; I'm pretty sure of that because I've seen it
working before.

-Karl

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Reconsidering the "buddy system"

[Karl Fogel <kf...@google.com>]

"Justin Erenkrantz" <ju...@erenkrantz.com> writes:
> On 8/8/06, C. Michael Pilato <cm...@collab.net> wrote:
>> We can avoid the patches-getting-dropped problem by simply having a single
>> person (perhaps on a rotating basis) watching the list for patches and
>> posting them to the tracker.  In fact, I thought we had already this --
>> Michael Thelen, where are you?
>
> If our patch manager has taken a siesta, it sounds reasonable to seek
> a replacement.  I think having a volunteer with the responsibility for
> incoming bug triage would resolve most of the user-facing concerns.
> And, if that person is up to it, they could also choose to remove the
> 'buddy system' - but it could really create a lot of duplicate issues
> - yet, if they're game to try to manage those dups, fine by me...  --

+1 on that idea -- a Bug Manager, in addition to a Patch Manager.

Max, thoughts?

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Reconsidering the "buddy system"

[Justin Erenkrantz <ju...@erenkrantz.com>]

On 8/8/06, C. Michael Pilato <cm...@collab.net> wrote:
> We can avoid the patches-getting-dropped problem by simply having a single
> person (perhaps on a rotating basis) watching the list for patches and
> posting them to the tracker.  In fact, I thought we had already this --
> Michael Thelen, where are you?

If our patch manager has taken a siesta, it sounds reasonable to seek
a replacement.  I think having a volunteer with the responsibility for
incoming bug triage would resolve most of the user-facing concerns.
And, if that person is up to it, they could also choose to remove the
'buddy system' - but it could really create a lot of duplicate issues
- yet, if they're game to try to manage those dups, fine by me...  --
justin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Reconsidering the "buddy system"

["C. Michael Pilato" <cm...@collab.net>]

I can't speak for the majority of the committers, but certainly the
CollabNet folks know what it's like to have our daily lives dictated by the
demands of the almight Issue Tracker.  That's just the way things are done
at CollabNet.  You *might* get a quick email or IRC fly-by asking if a
particular observed behavior was really a bug, but generally speaking, it's
straight to the tracker with any findings.

Now, things aren't as bad internally as they could be though.  Because there
are a fixed number of folks adding stuff to our internal tracker, and
because those folks have mostly been interacting with that tracker for
years, we don't see terribly many duplicate issues.  I suspect that
Subversion's tracker will be a whooole different ball of wax from that
perspective, but I could be wrong.

I say all that to say that if we decide to change our policy in the way you
suggest, that's fine with me, and I can attest to the fact that being
tracker-driven is actually a very productive way to be.

But that doesn't address the question of whether or not the Subversion
dev-folk *should* make that change.  I don't have a clear opinion of that,
really.

We can avoid the patches-getting-dropped problem by simply having a single
person (perhaps on a rotating basis) watching the list for patches and
posting them to the tracker.  In fact, I thought we had already this --
Michael Thelen, where are you?

The triage problem is a larger one, but here I have some questions, too.
So, a large part of my job is to do exactly this -- triage issues, decide
what's to be fixed and when, interact with the reporter to get more
information, estimated the effort and effects of the change, etc.  I can
only do this really well, though, when I have in my head the "big picture"
of where the product is going, and what milestones it aims to hit along the
way.  I'm not sure I could do such a thing for Subversion because, generally
speaking, our releases have very little definition.  It feels kinda like,
"Hey, it's been six months so we should release something."  That's not a
particularly inspired way to run a project, and certainly doesn't lend
itself to a clear picture of when particular defects or enhancements get
fielded.  Remember the final push to 1.0?  The definition of that release
was much more solid than really anything since, and I think triaging issues
(which we did quite a bit of back then) was easier to do.


Max Bowsher wrote:
> These days, the dev@ list is a very busy place. It takes a truly
> dedicated person to attempt to read everything that goes on there. I
> regularly find myself discarding some threads, in order to reserve
> enough time to participate effectively in others. In particular, I am
> concerned about people posting patches which slip past our collective
> radar and are lost in oblivion. For this reason, I wonder if we have
> done wrong in trying to direct patches to email and never issue-tracking.
> 
> 
> I've recently gone and tried to read our issue tracker policy from an
> outside perspective. The result of that is that I am now afraid that we
> are making unreasonable demands on what people must do before they
> report bugs to us. I've come to the conclusion that if I was to
> encounter Subversion's policy for the first time in a project that I was
> not strongly involved in, I would be fairly indignant at the hoops I was
> being asked to jump through.
> 
> 
> For the two reasons above, I would like to propose that the buddy policy
> be abolished forthwith.
> 
> 
> Naturally, this will require that we developers interact more with the
> issue tracker. However, I think this is something we MUST do anyway -
> there are currently 72 issues in the "---" milestone, which is supposed
> to be a marker of "this issue has not yet received initial triage". I
> think we have fallen into the trap of marginalizing the issue tracker to
> the point that even the developers aren't using it effectively, for the
> most part.
> 
> 
> So, in brief, here's what I think we should do about it:
> 
> 
> 1. Modify the IZ front page to:
> 
> a) politely request that people carefully consider whether they have a
> true bug, or a case of user error, or a reasonably complete code
> contribution, or a simple draft patch for discussion.
> 
> b) guide people to use the issue tracker for bugs or significant code
> contribution, or the mailing list otherwise.
> 
> c) strongly recommend that when filing an issue, they also post a link
> and a brief summary to dev@, explaining that this will greatly help in
> getting prompt responses. Emphasize that the list is the place for
> discussion, and the issue tracker works best for checkpointing summaries
> of the progress of discussions, not holding them itself.
> 
> 
> 2. Set up a simple list of boilerplate responses for closing misguided
> issue filings, so that we can deal with them with an extreme minimum of
> effort.
> 
> 
> 3. Have a policy of regularly reviewing "---"-milestoned issues, perhaps
> with an automated periodic mailing of open "---" issues to dev.
> 
> 
> 4. Try to use IZ to maintain a central database of things that need to
> be done, rather than keeping a great deal of that information on
> individual brains and inboxes.
> 
> 
> 
> I am hopeful that the end result of this will be to take some of the
> pressure off dev@ as the sole mechanism for reporting *and tracking*
> inbound bug reports, and allow us to make better use of our issue tracker.
> 
> Max.
> 
> 
> 
> 
> 


-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

RE: Reconsidering the "buddy system"

[Lieven Govaerts <lg...@mobsol.be>]

> -----Original Message-----
> From: Max Bowsher [mailto:maxb1@ukf.net] 
> 
> These days, the dev@ list is a very busy place. It takes a 
> truly dedicated person to attempt to read everything that 
> goes on there. I regularly find myself discarding some 
> threads, in order to reserve enough time to participate 
> effectively in others. In particular, I am concerned about 
> people posting patches which slip past our collective radar 
> and are lost in oblivion. For this reason, I wonder if we 
> have done wrong in trying to direct patches to email and 
> never issue-tracking.
 
Interesting topic, but I don't agree with your solution to 'reconsider the
buddy system' so do you mind I take the opportunity to look a bit more at
the problem? I already read the other replies from Karl, Michael and Justin
so in some parts of my mail I'm also commenting on their suggestions.


When a user has a question she thinks might be a bug in Subversion, she is
asked to:
1) reread the FAQ and the book
2) search through the existing issues
3) find a buddy (on users, dev, irc...) to report a new issue

While I think this approach can work pretty good, our implementation has
some problems:
1) The book isn't uptodate with svn 1.3
2) Nobody reads FAQ's
3) Not all issues have meaningful summaries
4) The issue tracker is also used for patch tracking. While I think this is
OK, a PATCH is now implemented as a separate issue type which makes it more
difficult the see what the patch is for or to estimate its priority and
impact.
5) The existing issue report only shows the open issues, not the ones that
are already fixed but not yet included in a release (same for searching
through summaries). Leads to: 'yeah, that's issue #1234' mails on users@.
6) While there are a lot of issues (defect, task, patch, feature) in the
database, a lot of the fixes made in the code are not reported in the issue
tracker. Leads to: 'yeah, we fixed that on trunk in r12345' mails on dev@.
This problem also triggers other problems outside the issue tracking domain:
- it takes a long time to create the changelist,
- it's equally difficult to make sure all issues/changes included in a
release are covered by regression tests,
- while evaluating the impact of an upgrade, a Subversion admin has to look
through the logs of trunk to see in more detail what has changed in a new
release.

Possible solutions:
1) Alter the standard issue tracker report to also show issues resolved on
trunk. Ideally a user can select her current Subversion version, so the
report would include all open issues and the ones solved in a release newer
then the one she's using.
2) Whenever a change in functionality or an issue that can be encountered by
users is committed on trunk, report the change in the issue tracker. Or,
even better, use the Test Driven Development approach: report an
issue/feature including the acceptance criteria, write an
acceptance(//regression) test, fix the issue. I've been working with Eric on
the 'merge eol propchanges' issue (I wrote the tests -> Eric wrote the code
in a few iterations) and it has worked pretty good.
3) Add a separate comment to an issue with specifics on how a functional
change is implemented, so it becomes easier to update the documentation.
4) Remove the PATCH issue type and define a patch keyword. Re-categorize all
existing PATCH entries to issue, feature, enhancement or task and add the
patch keyword. Or instead of a new keyword patch we can define a new state
'patch available'? I'm undecided on this one.

Solution 2 and 3 can only work if someone is prepared to dedicate some time
to the issue tracker. So I'm also +1 on defining an Issue(Bug?) Manager
role, separate from the current Patch Manager role. 

I have my doubts on what you mean with 'triaging issues', so I'll define
what I think a Issue Manager can do:
-> follow up the usage of the issue tracker: identify duplicates, contact
the reporter when the issue is unclear, fix summaries, reproduce the
issue...
-> initial prioritization of issues
-> trigger discussion on severe issues
-> summarize and report regularely (delta issues since last release, issues
to be fixed for next release ...)
-> agree on goals for next release (e.g. no P1's anymore, all devs fix one
issue, hold a bug squashing party ...)
-> create umbrella patches where helpful
-> motivate non-committers to submit patches for bugs or regression tests
(bite-sized keywords)

In a corporate environment I'd put 'assign issues to an iteration/milestone'
as a task for the Issue Manager, but I'm a bit sceptical on how that will
work in the Subversion project. Example: in the issue tracker there are now
4 unresolved P1 issues, opened somewhere in 2004/2005. Since these are P1's,
the Issue Manager would assign them to the next (fix) release. But, because
the I.M. lacks any formal authority, those issues will only get fixed when
developers volunteer to work on them. 
That's why I think triaging is a task for the whole developer group.

Concerning who does what, some of the tasks of an Issue Manager are already
done now by any developer or tester with some time to spare. If devs agree
(atleast partly) with the things I propose, then I'm volunteering to take on
the Issue Manager role for atleast the next 6 months. That'll give me some
time to try some approaches and at the same time you'll find out pretty soon
if I don't have time/interest to continue doing this.

regards,

Lieven.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org