[Proposal] Refining our Development Process

[Ferdinand Soethe <fe...@apache.org>]

The following borrows quite heavily from an earlier proposal called
"[Proposal] Development process and a stable trunk". I hope I can put
this across more clearly this time.

I'll try to make it short :-)

Different Approaches to Development
-----------------------------------

There are two major approaches to software development represented in
the this project at the moment:

- the evolutionary coding approach where concepts are developed while
  coding and

- the classic approach of developing and discussing a concept in
  theory that is then coded and refinded.

Both approaches may have certain merits that I'm not about to
discuss here. Instead I like us acknowledge that both approaches
can help us develop Forrest and that we need to create an environment
where they can co-exist with each other and the needs of people who
are using Forrest.


Different Users
---------------

From a user's perspective. I can see at
least three different approaches:

1. Very Early Adopters (VEA) will be happy to always work with the latest
   unreleased version to try all the new features before
   they are even released.

   They have the skills to fix things if need to and accept the high
   cost of maintaining a site as a price for always having
   access to the latest features for themselves (or their clients).

2. A second group, let's call them Early adopters (EA), is also
   prepared to try and use new features in their projects, but for some
   reason cannot run sites with a bleeding edge version.

   Typical reasons for this are:

   - Mission critical sites
   - clients that don't accept downtime
   - a limited budget to migrate projects along the stages of
     bleeding edge development or
   - lack of skills on the part of the EA,
     so that quick fixes of problems are not always possible.

3. A third group are users of released code (URC). For what ever
   reasons they are happy to use a released and documented version and
   have neither skill nor urge to use anything else.

As far as I can tell, we are currently doing ok for groups 1 and 3,
while group 2 is quite often unable to work with trunk in their
projects. As a result we are loosing valuable input and participation.

Different ways of Participation
-------------------------------

Fortunately we are a community with very different backgrounds, which
are often reflected in the different development approaches
described earlier.

While some of us develop concepts through coding and re-factoring
code, others prefer to approach a question from an architects point of view.

I think that it is import to understand that this difference is not
always a matter of choice. Some people (like me) are simply to slow to
read and understand code to join the first camp while others lack the
patience (or ability) to think in concepts only.

We should design a development process that will allow both to
follow developments, to participate and contribute.

How to achieve all that?
------------------------

My idea is actually quite simple and it basically extends the
white-board concept already in place.

I'll try to explain it in a few headlines:

1. Freedom of choice

  If a group of people get the itch to develop something for Forrest it
  should be up to them to choose the development method that works best
  for them.

2. Whiteboard

  Any new development starts in whiteboard and will not become part of
  trunk until it is internally released (will define that in a moment).

  Being in whiteboard means:

  - people are completely free in how they approach development, if and
    how they document it and how often they change their concepts.

  - they are encouraged to develop and document their concepts early and
    discuss it with the whole of the project (to make sure everybody
    agrees with their architectural views) but there is no obligation to
    do so.

  - up to the point of internal release committers and other developers
    are welcome to but not expected to follow discussions on such a
    development or to involve themselves in the development process.

  - clear indicators will help to tell group internal threads from normal
    project-wide discussion so that people can safely ignore group
    internal mails if they chose to do so.

3: Internal release

  When the group is satisfied with architecture, features and
  performance of their development they propose an 'internal release' to
  integrate their development into trunk (or move a plugIn out of the
  whiteboard).

  Minimum Requirements for internal release are:
  (this may need further discussion)


  - A well documented and stable architectural concept that the group is
    prepared to defend in a project wide discussion.

  - Stable code that is not mature enough not to break trunk

  - Basic low level practical documentation
    = how to install
    = what will break / what needs to be done to migrate
    = how do I use its features

  - A preparedness to answer silly questions from people who try to
    flesh out documentation in the next stage.

  The outcome of the application can be one of the following:

  - Acceptance

    The development is sound and complete and is considered to be a
    useful extension of Forrest that we are happy to support as part
    of Forrest. It will then be scheduled for the next release (see
    4. below)


  - Postponement

    The development is either incomplete or has other deficiencies
    (including poor design, performance etc.)

    It remains in whiteboard for improvement and can once again be
    proposed for internal release later on.

  - Rejection because the proposal violates basic design concepts, adds
    blinking text or is deemed useless by the project (add other
    reasons).

    It may remain in whiteboard for everybody to use or give it another
    try.

4. External Release

  Developments accepted in the internal release process should be
  scheduled for release as soon as possible.

  As a rule we should not wait to accumulate several features but aim
  to release each feature on its own so that users can familiarize
  themselves with and test each new feature as soon as possible.

  The idea is that integrating, testing and documenting an already
  well developed new feature will allow a great number of people to
  involve themselves (in integration, testing, documentation) and
  could perhaps become a subject for a Forrest Friday with many
  different tasks that can be worked on in paralell.

  I'd also like to see this develop as phases in the project year
  where many committers work together on a common goal and get familiar
  with each others work.


  
So, wdyt?
Ferdinand
  



  


  
  




   







--
Ferdinand Soethe

Re: [Proposal] Refining our Development Process

[Ross Gardler <rg...@apache.org>]

Ferdinand Soethe wrote:
> [I'm sipping an excellent yeast brandy while typing this so please
>  excuse if not everything makes perfect sense :-)]
> 
> Ross Gardler wrote:
> 
> 
>>In open source projects there is a fourth type of user. This one sits
>>somewhere between your type 1 and type 2, lets call them type "one and a
>>half".
> 
> 
>>These people are very technical and are able to work on experimental 
>>parts of the code base where that code adds benefit to their project. 
>>However, they may not want to work with other experimental parts of the
>>code base.
> 
> 
> Thanks for adding that piece of info.
> So what exactly is the difference between group 1 and 1.5?

A group 1 person will be happy to work with all experimental code. 
Perhaps they do not have any mission critical applications using the 
code base or they are between projects and have some time to experiment.

A type 1.5 needs some new functionality for a mission critical app and 
will therefore spend time working on that piece of functionality, but 
they will not want to expose the app to non-essential development code.

Ross

Re: [Proposal] Refining our Development Process

[Ferdinand Soethe <fe...@apache.org>]

[I'm sipping an excellent yeast brandy while typing this so please
 excuse if not everything makes perfect sense :-)]

Ross Gardler wrote:

> In open source projects there is a fourth type of user. This one sits
> somewhere between your type 1 and type 2, lets call them type "one and a
> half".

> These people are very technical and are able to work on experimental 
> parts of the code base where that code adds benefit to their project. 
> However, they may not want to work with other experimental parts of the
> code base.

Thanks for adding that piece of info.
So what exactly is the difference between group 1 and 1.5?

>> As far as I can tell, we are currently doing ok for groups 1 and 3,
>> while group 2 is quite often unable to work with trunk in their
>> projects. As a result we are loosing valuable input and participation.

> Agreed. The plugin system should make it possible to accomodate the type
> "one and a half" people. Just develop new experimental code as a plugin
> and leave it up to the adopters to choose whether to use it or not. This
> can work well, witness the fact that dispatcher work, in the main, did
> not affect users of core.

Hmm. So are these 1.5 people early adopters that are skilled and
interested to work with parts of a new release like Diwaker did by
implementing a dispatcher skin?

> Let me expand on that a little - new *feature* development is in a
> whiteboard plugin, or if this is not possible in a branch.

Thanks for adding that. I fully agree.

> Firstly:

> The strength of Open Source development is the many eyes concept. To 
> encourage people to ignore parts of the development we are removing 
> this.

Let's say we are simply trying to be realistic about this. As it is
people will opt out of certains discussions for lack of time, or
interest or other reasons.

By separating the (high volume) discussion of early development phases
from the rest of our discussions I hope that we'll be more focussed on
the important stuff. One of them being the discussion when a group
asks the project for comments or an internal release.

> We need to be very careful what we mean by "safely ignore". From a
>   quality perspective we cannot "safely ignore" since important issues
> may be missed.

I see it from a different perspective: Having that virtual space of
their own, will allow a group to make AND correct their own mistakes
which is an important part of the learning process.

If they follow the advice to discuss their concepts with the project
early, nothing is lost. If they don't, flaws will surface in the
internal release phase and they may have wasted some time in coding
something that will never make it into core. That also can be a
learning process.


> PMC oversight is requried to ensure that there are no legal issues with
> new code being developed. If we "safely ignore" development on some 
> aspect we are possibly ignoring critical issues.

True. This is one question we really need to monitor.

otoh the only people who can commit code are committers. And they should be
well aware of license issues. So the least we could expect from them
(us) is to ask unless a licensing situation is clear.

> That being said, the
> reality is that no single member of the PMC reads every single commit 
> message or email anyway. So again, I think the problem is in the 
> terminology rather than your intent.

Ok, but this part definitely needs refinement.

> Thirdly:

> It should be reinforced that if someone decides not to particpate in 
> development of a new feature within the whiteboard there opinions in 
> terms of arcitecture and design are still valid. One does not have to 
> contribute code to contribute to a development effort. We have seen that
> it can be upsetting for developers creating new code when they get 
> feedback that generates yet more work, but they get little in the way of
> code contributions. This is to be expected from the "one and a half" 
> people since they have a deep understanding of the core system and its
> direction, but little time to contribute to every part of development.

Thanks, can't stress that enough.


> However, even though everyone is entitled to an opinion, it does not 
> mean the whiteboard developers have to impelment it in line with that 
> opinion. If their itch is different then fair enough.

Let me add: Yes! You can still ignore and reject such 'parachuted'
advice, but you have to be pretty confident about the quality of your
design. Or else it will come back to haunt you during the internal
release discussions.

>>   - Stable code that is not mature enough not to break trunk

> You mean that *is* mature enough not to break trunk.

Correct. Thanks for catching this.

>>   - Basic low level practical documentation
>>     = how to install
>>     = what will break / what needs to be done to migrate
>>     = how do I use its features
>> 
>>   - A preparedness to answer silly questions from people who try to
>>     flesh out documentation in the next stage.

> lets add:

> A suite of tests. For plugins that are well documented we have tests 
> built into the documentation. That is if the docs have examples of all
> features, then doing "ant test" in the plugin directory will run the 
> tests accordingly.

+1

>>   The outcome of the application can be one of the following:

[...]

> OK, this is all fine and can be covered by a simple majority vote after
> discussing the proposal.

True.

My point here was that a postponement or rejection does
not mean that it cannot be used or worked on. It only means that it
will not become part of an official Forrest release that we all
support.

It's all about allowing space for spontaneous and creative development
within the project but outside of the official releases.

> I like it.

Thanks for your comments.

--
Ferdinand Soethe

Re: [Proposal] Refining our Development Process

[David Crossley <cr...@apache.org>]

Ross Gardler wrote:
> Ferdinand Soethe wrote:
> >The following borrows quite heavily from an earlier proposal called
> >"[Proposal] Development process and a stable trunk". I hope I can put
> >this across more clearly this time.
> >
> >I'll try to make it short :-)

Thanks for trying to move this forward.

Adding a note for the email archives: 
See links to other discussions at: 
http://forrest.apache.org/guidelines.html#develop
http://issues.apache.org/jira/browse/FOR-631

> >Different Approaches to Development
> >-----------------------------------
> >
> >There are two major approaches to software development represented in
> >the this project at the moment:
> >
> >- the evolutionary coding approach where concepts are developed while
> >  coding and
> >
> >- the classic approach of developing and discussing a concept in
> >  theory that is then coded and refinded.
> 
> ...
> 
> >Different Users
> >---------------
> >
> >From a user's perspective. I can see at
> >least three different approaches:
> >
> >1. Very Early Adopters (VEA) will be happy to always work with the latest
> >   unreleased version to try all the new features before
> >   they are even released.
> 
> ...
> 
> >2. A second group, let's call them Early adopters (EA), is also
> >   prepared to try and use new features in their projects, but for some
> >   reason cannot run sites with a bleeding edge version.
> 
> ...
> 
> >3. A third group are users of released code (URC). For what ever
> >   reasons they are happy to use a released and documented version and
> >   have neither skill nor urge to use anything else.
> 
> 
> In open source projects there is a fourth type of user. This one sits 
> somewhere between your type 1 and type 2, lets call them type "one and a 
> half".
> 
> These people are very technical and are able to work on experimental 
> parts of the code base where that code adds benefit to their project. 
> However, they may not want to work with other experimental parts of the 
> code base.
> 
> >As far as I can tell, we are currently doing ok for groups 1 and 3,
> >while group 2 is quite often unable to work with trunk in their
> >projects. As a result we are loosing valuable input and participation.
> 
> Agreed. The plugin system should make it possible to accomodate the type 
> "one and a half" people. Just develop new experimental code as a plugin 
> and leave it up to the adopters to choose whether to use it or not. This 
> can work well, witness the fact that dispatcher work, in the main, did 
> not affect users of core.
> 
> >Different ways of Participation
> >-------------------------------
> 
> ...
> 
> >
> >We should design a development process that will allow both to
> >follow developments, to participate and contribute.
> 
> That's a tall order, lets see how well you do...
> 
> >How to achieve all that?
> >------------------------
> >
> >My idea is actually quite simple and it basically extends the
> >white-board concept already in place.
> >
> >I'll try to explain it in a few headlines:
> >
> >1. Freedom of choice
> >
> >  If a group of people get the itch to develop something for Forrest it
> >  should be up to them to choose the development method that works best
> >  for them.
> >
> >2. Whiteboard
> >
> >  Any new development starts in whiteboard and will not become part of
> >  trunk until it is internally released (will define that in a moment).
> 
> Let me expand on that a little - new *feature* development is in a 
> whiteboard plugin, or if this is not possible in a branch.

Regarding Ferdinand's comment: "starts in whiteboard and
will not become part of trunk until it is internally released":
The whiteboard already is part of trunk, and it
goes out with a release.

I cannot yet see how, but perhaps "whiteboard" can
be outside of trunk in our SVN repository. I think
that i would prefer to *not* do that. Shipping the
whiteboard along with the release as we do currently,
probably gives more attention to those plugins.

> >  Being in whiteboard means:
> >
> >  - people are completely free in how they approach development, if and
> >    how they document it and how often they change their concepts.
> >
> >  - they are encouraged to develop and document their concepts early and
> >    discuss it with the whole of the project (to make sure everybody
> >    agrees with their architectural views) but there is no obligation to
> >    do so.
> >
> >  - up to the point of internal release committers and other developers
> >    are welcome to but not expected to follow discussions on such a
> >    development or to involve themselves in the development process.
> >
> >  - clear indicators will help to tell group internal threads from normal
> >    project-wide discussion so that people can safely ignore group
> >    internal mails if they chose to do so.
> 
> There are a few problems with this:
> 
> Firstly:
> 
> The strength of Open Source development is the many eyes concept. To 
> encourage people to ignore parts of the development we are removing 
> this. We need to be very careful what we mean by "safely ignore". From a 
>  quality perspective we cannot "safely ignore" since important issues 
> may be missed. However, since everything is in a plugin and that plugin 
> may never make it out of the whiteboard we are protecting the core 
> project. So I'll play along for now - I'm interested in seeing your 
> proposal for migration out of the whiteboard.

I too am concerned. If something stays in the whiteboard
for a long time, and people have been ignoring its active
development, and only assess it when it is time for the
internal release, then fundamental issue might arise.

> Secondly:
> 
> PMC oversight is requried to ensure that there are no legal issues with 
> new code being developed.

It is more than that. The PMC needs to also oversee the
community and the code. With the code, if a PMC member
sees it going in a direction that might contravene the
project goals, or might cause a maintenance nightmare,
or adversly affect other parts, then they need to speak up
as soon as possible. This is defined at "Lazy approval":
http://forrest.apache.org/guidelines.html#approvals
http://forrest.apache.org/guidelines.html#actions
It is very important that people understand:
http://www.apache.org/foundation/how-it-works.html#committers
"they are actually making short-term decisions for the project.
The PMC can (even tacitly) agree and approve it into permanency,
or they can reject it. Remember that the PMC makes the decisions,
not the individual people."

So i agree that we need to be very careful not to
encourage people to ignore stuff.

> If we "safely ignore" development on some 
> aspect we are possibly ignoring critical issues. That being said, the 
> reality is that no single member of the PMC reads every single commit 
> message or email anyway. So again, I think the problem is in the 
> terminology rather than your intent.

With a small PMC we do have a problem in this regard.

> Thirdly:
> 
> It should be reinforced that if someone decides not to particpate in 
> development of a new feature within the whiteboard there opinions in 
> terms of arcitecture and design are still valid. One does not have to 
> contribute code to contribute to a development effort. We have seen that 
> it can be upsetting for developers creating new code when they get 
> feedback that generates yet more work, but they get little in the way of 
> code contributions. This is to be expected from the "one and a half" 
> people since they have a deep understanding of the core system and its 
> direction, but little time to contribute to every part of development.
> 
> However, even though everyone is entitled to an opinion, it does not 
> mean the whiteboard developers have to impelment it in line with that 
> opinion. If their itch is different then fair enough.

Unless there has been a "Lazy approval" -1 from a
PMC member, at which point we need to make a decision.
That should be able to happen at any stage, otherwise
when it comes to internal release time, our concerns
might have escalated or been forgotten.

> >3: Internal release
> >
> >  When the group is satisfied with architecture, features and
> >  performance of their development they propose an 'internal release' to
> >  integrate their development into trunk (or move a plugIn out of the
> >  whiteboard).
> >
> >  Minimum Requirements for internal release are:
> >  (this may need further discussion)
> >
> >
> >  - A well documented and stable architectural concept that the group is
> >    prepared to defend in a project wide discussion.
> >
> >  - Stable code that is not mature enough not to break trunk
> 
> You mean that *is* mature enough not to break trunk.
> 
> >  - Basic low level practical documentation
> >    = how to install
> >    = what will break / what needs to be done to migrate
> >    = how do I use its features
> >
> >  - A preparedness to answer silly questions from people who try to
> >    flesh out documentation in the next stage.
> 
> lets add:
> 
> A suite of tests. For plugins that are well documented we have tests 
> built into the documentation. That is if the docs have examples of all 
> features, then doing "ant test" in the plugin directory will run the 
> tests accordingly.

That is very important.

For some some plugins it is hard to have tests for
everything. Having demonstrations on the zone server
helps. We should probably have builds of our site-author
docs happening there, configured for both "skins" and
"dispatcher".

> >  The outcome of the application can be one of the following:
> >
> >  - Acceptance
> >
> >    The development is sound and complete and is considered to be a
> >    useful extension of Forrest that we are happy to support as part
> >    of Forrest. It will then be scheduled for the next release (see
> >    4. below)

Defining "complete" is difficult.

> >  - Postponement
> >
> >    The development is either incomplete or has other deficiencies
> >    (including poor design, performance etc.)
> >
> >    It remains in whiteboard for improvement and can once again be
> >    proposed for internal release later on.
> >
> >  - Rejection because the proposal violates basic design concepts, adds
> >    blinking text or is deemed useless by the project (add other
> >    reasons).
> >
> >    It may remain in whiteboard for everybody to use or give it another
> >    try.
> 
> OK, this is all fine and can be covered by a simple majority vote after 
> discussing the proposal.

Please use the terms defined at:
http://forrest.apache.org/guidelines.html#approvals
I presume that you mean "Lazy majority".

> >4. External Release
> >
> >  Developments accepted in the internal release process should be
> >  scheduled for release as soon as possible.
> >
> >  As a rule we should not wait to accumulate several features but aim
> >  to release each feature on its own so that users can familiarize
> >  themselves with and test each new feature as soon as possible.
> 
> In the case of plugins this is no problem since they can now have an 
> independant release cycle.
> 
> >So, wdyt?
> 
> I like it.

I am not sure yet.

Because whiteboard plugins are delivered with a
release, people will use them. I presume that we
are not binding ourselves to any backward-compatibility.

What happens with core plugins that need to have
radical changes, such as PhotoGallery plugin recently.
Bumping the version number helped, because users could
declare the previous version in their forrest.properties
However for users of trunk, it was suddenly broken.

The Dispatcher used a technique of constantly changing
the plugin name, which seemed to be disruptive.
Yes, i know that its development was difficult and there
was to-and-fro about using a branch, etc. I am trying
to explore how is the best way to continue development
of plugins.

I reckon that we also need to re-read our other past
discussions, especially Nicola Ken's "Project participation
and hackability" threads. See:
http://forrest.apache.org/guidelines.html#develop

-David

Re: [Proposal] Refining our Development Process

[Ross Gardler <rg...@apache.org>]

Ferdinand Soethe wrote:
> The following borrows quite heavily from an earlier proposal called
> "[Proposal] Development process and a stable trunk". I hope I can put
> this across more clearly this time.
> 
> I'll try to make it short :-)
> 
> Different Approaches to Development
> -----------------------------------
> 
> There are two major approaches to software development represented in
> the this project at the moment:
> 
> - the evolutionary coding approach where concepts are developed while
>   coding and
> 
> - the classic approach of developing and discussing a concept in
>   theory that is then coded and refinded.

...

> Different Users
> ---------------
> 
> From a user's perspective. I can see at
> least three different approaches:
> 
> 1. Very Early Adopters (VEA) will be happy to always work with the latest
>    unreleased version to try all the new features before
>    they are even released.

...

> 2. A second group, let's call them Early adopters (EA), is also
>    prepared to try and use new features in their projects, but for some
>    reason cannot run sites with a bleeding edge version.

...

> 3. A third group are users of released code (URC). For what ever
>    reasons they are happy to use a released and documented version and
>    have neither skill nor urge to use anything else.


In open source projects there is a fourth type of user. This one sits 
somewhere between your type 1 and type 2, lets call them type "one and a 
half".

These people are very technical and are able to work on experimental 
parts of the code base where that code adds benefit to their project. 
However, they may not want to work with other experimental parts of the 
code base.

> As far as I can tell, we are currently doing ok for groups 1 and 3,
> while group 2 is quite often unable to work with trunk in their
> projects. As a result we are loosing valuable input and participation.

Agreed. The plugin system should make it possible to accomodate the type 
"one and a half" people. Just develop new experimental code as a plugin 
and leave it up to the adopters to choose whether to use it or not. This 
can work well, witness the fact that dispatcher work, in the main, did 
not affect users of core.

> Different ways of Participation
> -------------------------------

...

> 
> We should design a development process that will allow both to
> follow developments, to participate and contribute.

That's a tall order, lets see how well you do...

> How to achieve all that?
> ------------------------
> 
> My idea is actually quite simple and it basically extends the
> white-board concept already in place.
> 
> I'll try to explain it in a few headlines:
> 
> 1. Freedom of choice
> 
>   If a group of people get the itch to develop something for Forrest it
>   should be up to them to choose the development method that works best
>   for them.
> 
> 2. Whiteboard
> 
>   Any new development starts in whiteboard and will not become part of
>   trunk until it is internally released (will define that in a moment).

Let me expand on that a little - new *feature* development is in a 
whiteboard plugin, or if this is not possible in a branch.

>   Being in whiteboard means:
> 
>   - people are completely free in how they approach development, if and
>     how they document it and how often they change their concepts.
> 
>   - they are encouraged to develop and document their concepts early and
>     discuss it with the whole of the project (to make sure everybody
>     agrees with their architectural views) but there is no obligation to
>     do so.
> 
>   - up to the point of internal release committers and other developers
>     are welcome to but not expected to follow discussions on such a
>     development or to involve themselves in the development process.
> 
>   - clear indicators will help to tell group internal threads from normal
>     project-wide discussion so that people can safely ignore group
>     internal mails if they chose to do so.

There are a few problems with this:

Firstly:

The strength of Open Source development is the many eyes concept. To 
encourage people to ignore parts of the development we are removing 
this. We need to be very careful what we mean by "safely ignore". From a 
  quality perspective we cannot "safely ignore" since important issues 
may be missed. However, since everything is in a plugin and that plugin 
may never make it out of the whiteboard we are protecting the core 
project. So I'll play along for now - I'm interested in seeing your 
proposal for migration out of the whiteboard.

Secondly:

PMC oversight is requried to ensure that there are no legal issues with 
new code being developed. If we "safely ignore" development on some 
aspect we are possibly ignoring critical issues. That being said, the 
reality is that no single member of the PMC reads every single commit 
message or email anyway. So again, I think the problem is in the 
terminology rather than your intent.

Thirdly:

It should be reinforced that if someone decides not to particpate in 
development of a new feature within the whiteboard there opinions in 
terms of arcitecture and design are still valid. One does not have to 
contribute code to contribute to a development effort. We have seen that 
it can be upsetting for developers creating new code when they get 
feedback that generates yet more work, but they get little in the way of 
code contributions. This is to be expected from the "one and a half" 
people since they have a deep understanding of the core system and its 
direction, but little time to contribute to every part of development.

However, even though everyone is entitled to an opinion, it does not 
mean the whiteboard developers have to impelment it in line with that 
opinion. If their itch is different then fair enough.

> 3: Internal release
> 
>   When the group is satisfied with architecture, features and
>   performance of their development they propose an 'internal release' to
>   integrate their development into trunk (or move a plugIn out of the
>   whiteboard).
> 
>   Minimum Requirements for internal release are:
>   (this may need further discussion)
> 
> 
>   - A well documented and stable architectural concept that the group is
>     prepared to defend in a project wide discussion.
> 
>   - Stable code that is not mature enough not to break trunk

You mean that *is* mature enough not to break trunk.

>   - Basic low level practical documentation
>     = how to install
>     = what will break / what needs to be done to migrate
>     = how do I use its features
> 
>   - A preparedness to answer silly questions from people who try to
>     flesh out documentation in the next stage.

lets add:

A suite of tests. For plugins that are well documented we have tests 
built into the documentation. That is if the docs have examples of all 
features, then doing "ant test" in the plugin directory will run the 
tests accordingly.

>   The outcome of the application can be one of the following:
> 
>   - Acceptance
> 
>     The development is sound and complete and is considered to be a
>     useful extension of Forrest that we are happy to support as part
>     of Forrest. It will then be scheduled for the next release (see
>     4. below)
> 
> 
>   - Postponement
> 
>     The development is either incomplete or has other deficiencies
>     (including poor design, performance etc.)
> 
>     It remains in whiteboard for improvement and can once again be
>     proposed for internal release later on.
> 
>   - Rejection because the proposal violates basic design concepts, adds
>     blinking text or is deemed useless by the project (add other
>     reasons).
> 
>     It may remain in whiteboard for everybody to use or give it another
>     try.

OK, this is all fine and can be covered by a simple majority vote after 
discussing the proposal.

> 4. External Release
> 
>   Developments accepted in the internal release process should be
>   scheduled for release as soon as possible.
> 
>   As a rule we should not wait to accumulate several features but aim
>   to release each feature on its own so that users can familiarize
>   themselves with and test each new feature as soon as possible.

In the case of plugins this is no problem since they can now have an 
independant release cycle.

> So, wdyt?

I like it.

Ross

Re: [Proposal] Refining our Development Process

[David Crossley <cr...@apache.org>]

Ross Gardler wrote:
> David Crossley wrote:
> >Ferdinand Soethe wrote:
> >>  Whiteboard and Peer Review Priciples
> >>    
> >>  It's important to point out that this way of using the whiteboard
> >>  is an attempt to deal with limited resources in a realistic way by
> >>  allowing committers with limited time, to 'savely ignore' the
> >>  discussions of early development stages and focus on paticipation
> >>  in later stages (or when they are explicitly asked for an opinion).
> >
> >I, for one, am not prepared to add the concept of
> >"safely ignore" to our project guidelines. It totally
> >goes against the Apache projects' PMC oversight principles.
> >
> >People can make up their own mind about what they follow
> >and what they don't follow. I don't see a need to explictly
> >sanction the ignoring of topics. KISS.
> 
> I think I also voiced my dislike of "safely ignore" in response to the 
> first draft. It is the duty of the PMC to provide oversight. In my 
> previous response I acknowledged that the reality is that not every PMC 
> member reads every mail and commit but, we have enough eyes to ensure 
> nothing is ignored. There is no such thing as "safely ignoring" within a 
> community project - only freedom of choice.
> 
> Why are the words important - because it removes the responsability, and 
> it is that responsability that ensures we find what time we can to 
> provide the necessary oversight.
> 
> -1 to "safely ignore"
> +1 to freedom to choose
> 
> >> Minimum Requirements for internal release are:
> >> (this may need further discussion)
> >
> >
> >Also needs to gain "community". If the code is a
> >one-person affair then we will get into trouble.
> >I suppose that an actual Vote would express that,
> >because +1 also indicates a willingness to assist.
> 
> Yes, and we need to stress that in the vote. +0 will help assist a 
> projects "release" but not provide false promises of contribuions, or 
> indicate false support.
> 
> One thing I am guilty of is saying +1 when I should be saying +0. I (and 
> we) must stop this.

This would work if we are "voting". However, IIUC
this current proposal uses Lazy Consensus, i.e. no vote.

More and more i like the idea of doing an explicit
vote for moving plugins out of the whiteboard.

> >It should be stressed that stuff should move out of
> >the whiteboard as soon as possible (similar to how
> >branches should merge ASAP). Don't stay there until
> >it is complete.
> >
> >What level of functionality is required? Just basic
> >i presume.
> 
> I would say, "just basic". A community of support is more important. 
> Look at the Daisy plugin as an example. It is feature rich and very 
> complete. However, to maintain it requries a fairly in-depth knowledge 
> of Daisy. I don't think anyone here has that knowledge.
> 
> On the other hand, something like the photogallery isn't very feature 
> rich but it has had three different commiters and at least one other is 
> using it. That should probably come out of the whiteboard.
> 
> >>The outcome of the application can be one of the following (decided by
> >>a Lazy approval [1] after discussing the proposal):
> >
> >
> >Using "Lazy approval" is intended for the normal
> >development process, i.e. just do it, and if someone
> >raises a alarm bell then we backtrack/fix/decide.
> >This encourages efficient development and is used
> >as often as possible.
> >
> >With your proposed change of process i do not see how
> >it will work.
> >
> >Using lazy approval makes it hard to get the beast
> >back into the whiteboard in the case of "Postponement"
> >or "Rejection".
> 
> yes, given the need for community we need a minimum number of "+1" votes.
> 
> ...
> 
> >There are some other things that need considering ...
> >
> >There is already a concept about "deployed" plugins
> >and "released" plugins. I cannot see where this is
> >defined in our docs.
> 
> It is not defined in our docs. I won'r go into it here, but we need to 
> define them

There is already discussion about it in the archives,
and in plugins/build.xml
deploy - Deploy the unversioned plugin to the website SVN
release - Release the versioned plugin to the website SVN

> (and indeed decide if we still need deployed plugins now we 
> have auto deployment from the src directory).

Interesting idea. Lets talk about that in a separate topic.

> >Also we haven't yet defined a process for official releases
> >of plugins as separate from the release of a whole package
> >of trunk (e.g. forrest-0.7) which includes all plugins
> >(including whiteboard) in their current state. Official
> >releases of any product requires a PMC vote.
> 
> This should not be a blocker for Ferdinands proposal. As I see it this 
> proposal is about moving from whiteboard. It is not really about 
> releasing a versin of a plugin. That is, it could still be an unreleased 
> plugin, just one that is in core.

That is one reason i said that, so that we 
clearly state that there are various steps for
developing and releasing a plugin.

> >That leads to another problem with the current proposal.
> >When it comes time for a release of Forrest (e.g. 0.8)
> >then the PMC needs to vote on what is packaged with that
> >release. Currently that includes $FORREST_HOME/whiteboard/.
> >PMC members who have not been watching what is happening
> >would have difficulty with voting.
> 
> As I proposed befor the last release, we shoould leave all plugins out 
> of the binary release. Plugins are auto-downloaded when they are need, 
> why distribute them?

Wow, sorry i missed that suggestion.
Lets talk about that separately.

-David

> We can provide separate pacakges for whiteboard and plugins to assist 
> those trying to work offline.
>
> >I wondered earlier in this thread if whiteboard should
> >be moved outside of our trunk. That might solve some
> >problems, but there are other factors against doing it.
> 
> There you go ;-) (replying as I read - my previous para takes this idea 
> one step further)
> 
> Ross

Re: [Proposal] Refining our Development Process

[Ross Gardler <rg...@apache.org>]

David Crossley wrote:
>>   Whiteboard and Peer Review Priciples
>>     
>>   It's important to point out that this way of using the whiteboard
>>   is an attempt to deal with limited resources in a realistic way by
>>   allowing committers with limited time, to 'savely ignore' the
>>   discussions of early development stages and focus on paticipation
>>   in later stages (or when they are explicitly asked for an opinion).
> 
> 
> I, for one, am not prepared to add the concept of
> "safely ignore" to our project guidelines. It totally
> goes against the Apache projects' PMC oversight principles.
> 
> People can make up their own mind about what they follow
> and what they don't follow. I don't see a need to explictly
> sanction the ignoring of topics. KISS.

I think I also voiced my dislike of "safely ignore" in response to the 
first draft. It is the duty of the PMC to provide oversight. In my 
previous response I acknowledged that the reality is that not every PMC 
member reads every mail and commit but, we have enough eyes to ensure 
nothing is ignored. There is no such thing as "safely ignoring" within a 
community project - only freedom of choice.

Why are the words important - because it removes the responsability, and 
it is that responsability that ensures we find what time we can to 
provide the necessary oversight.

-1 to "safely ignore"
+1 to freedom to choose

>>  Minimum Requirements for internal release are:
>>  (this may need further discussion)
> 
> 
> Also needs to gain "community". If the code is a
> one-person affair then we will get into trouble.
> I suppose that an actual Vote would express that,
> because +1 also indicates a willingness to assist.

Yes, and we need to stress that in the vote. +0 will help assist a 
projects "release" but not provide false promises of contribuions, or 
indicate false support.

One thing I am guilty of is saying +1 when I should be saying +0. I (and 
we) must stop this.

> It should be stressed that stuff should move out of
> the whiteboard as soon as possible (similar to how
> branches should merge ASAP). Don't stay there until
> it is complete.
> 
> What level of functionality is required? Just basic
> i presume.

I would say, "just basic". A community of support is more important. 
Look at the Daisy plugin as an example. It is feature rich and very 
complete. However, to maintain it requries a fairly in-depth knowledge 
of Daisy. I don't think anyone here has that knowledge.

On the other hand, something like the photogallery isn't very feature 
rich but it has had three different commiters and at least one other is 
using it. That should probably come out of the whiteboard.

>>The outcome of the application can be one of the following (decided by
>>a Lazy approval [1] after discussing the proposal):
> 
> 
> Using "Lazy approval" is intended for the normal
> development process, i.e. just do it, and if someone
> raises a alarm bell then we backtrack/fix/decide.
> This encourages efficient development and is used
> as often as possible.
> 
> With your proposed change of process i do not see how
> it will work.
> 
> Using lazy approval makes it hard to get the beast
> back into the whiteboard in the case of "Postponement"
> or "Rejection".

yes, given the need for community we need a minimum number of "+1" votes.

...

> There are some other things that need considering ...
> 
> There is already a concept about "deployed" plugins
> and "released" plugins. I cannot see where this is
> defined in our docs.

It is not defined in our docs. I won'r go into it here, but we need to 
define them (and indeed decide if we still need deployed plugins now we 
have auto deployment from the src directory).

> Also we haven't yet defined a process for official releases
> of plugins as separate from the release of a whole package
> of trunk (e.g. forrest-0.7) which includes all plugins
> (including whiteboard) in their current state. Official
> releases of any product requires a PMC vote.

This should not be a blocker for Ferdinands proposal. As I see it this 
proposal is about moving from whiteboard. It is not really about 
releasing a versin of a plugin. That is, it could still be an unreleased 
plugin, just one that is in core.

> That leads to another problem with the current proposal.
> When it comes time for a release of Forrest (e.g. 0.8)
> then the PMC needs to vote on what is packaged with that
> release. Currently that includes $FORREST_HOME/whiteboard/.
> PMC members who have not been watching what is happening
> would have difficulty with voting.

As I proposed befor the last release, we shoould leave all plugins out 
of the binary release. Plugins are auto-downloaded when they are need, 
why distribute them?

We can provide separate pacakges for whiteboard and plugins to assist 
those trying to work offline.

> I wondered earlier in this thread if whiteboard should
> be moved outside of our trunk. That might solve some
> problems, but there are other factors against doing it.

There you go ;-) (replying as I read - my previous para takes this idea 
one step further)

Ross

Re: [Proposal] Refining our Development Process

[Ferdinand Soethe <fe...@apache.org>]

Thanks for your comments.

Will need more time to read both your responses in a quiet moment so
I'll postpone putting this to a vote until next week.

--
Ferdinand Soethe

Re: [Proposal] Refining our Development Process

[David Crossley <cr...@apache.org>]

Ferdinand Soethe wrote:
> David Crossley wrote:
> 
> > Ferdinand Soethe wrote:
> >> Since the debate on this seems to have died down, I have changed and
> >> extended some parts of this paper to address the points raised by
> >> David and Ross.
> >> 
> >> Now I would like to invite all for a last round of refinement before I
> >> put this to a vote on Friday.
> >
> > When you do, please change the Subject line so that
> > it explictly says "whiteboard".
> 
> I'd rather not do that because the changes go beyond the whiteboard
> process and so a change in subject line might be misleading.

I was happier when this current proposal was
limited to just the whiteboard. It was a manageable
chunk and the rest could be addressed later.
http://forrest.apache.org/guidelines.html#development

So i suppose that i agree with your next point
about "refining".

The trouble is, if you are going to call a vote
then we need something specific to vote on.

> > There are many other
> > aspects that have not been addressed by this proposal,
> > so it does not apply to other development (non-whiteboard).
> 
> I thought "refining" reflected my attempt to improve on some parts
> rather than re-writing it completely.
> Perhaps we can find a better wording for this.
> 
> > http://forrest.apache.org/guidelines.html#development
> > e.g. define how we add new functionality to plugins that
> > have already emerged from the whiteboard and how to
> > maintain a trunk that is "always usable".
> 
> > Another thing that i would like to say up front is that
> > i find myself repeating stuff said in the last major
> > discussions. I would encourage people to read this
> > whole thread:
> > [A]
> >  [Proposal] Development process and a stable trunk
> >  http://marc.theaimsgroup.com/?t=112504310100005
> 
> I'm certainly guilty of ignoring that thread when I wrote this
> proposal. One of the reasons being that I got the impression that some
> of the responses then were written on the bases of misunderstandings,
> so that I wanted to get a fresh start.

I read that thread and didn't see fundamental
misunderstandings. I will try again.

I thought that everything was clarified and we
just needed to document it. We didn't, so here we
go again.
 
> > What is it that stops group 2 from operating?
> > Why can't they use a particular SVN revision number
> > of trunk that they know works?
> 
> [Sorry Thorsten, I know this will sound like Structurer-bashing
>  again but it is such a good example. ]
> 
> The problem is that our 'stable trunk policy' _solved_
> the problem of projects failing to work for broken code reasons
> 
> but it _did not solve_ the problem of projects breaking because of
> repeated changes in the architecture of new features.
> 
> And it also did not solve the problem of people investing a lot of
> time and effort in understanding an quickly evolving architectural
> concept.
> 
> In other words I todays think that I'm glad that I didn't even
> try and learn views or earlier structurer versions because I would
> have wasted my time and others (by asking detailed questions like I do
> ask now about the locationmaps).
> 
> But since this not always obvious at the time it was trying to
> establish a process that more clearly marks a borderline between wild
> creative hacking style development and a mature concept.

Ah, thanks for explaining that. It helps a lot.

> > As i said in the recent discussion, "whiteboard" is already
> > in the "trunk". I think that you mean "move out of whiteboard".
> 
> Yes, sorry, I missed that sentence when I edited my paper.
> 
> > Tim, and all other respondents, raised some very
> > important concerns in the previous proposal [A].
> 
> Sorry, I'm not very good at deciding which ones of those comments are
> still valid for the current proposal. But since you pointed to Tims
> concerns I'm happy to requote and address them here:

The main thing that i was referring to was our
previous comments about "safely ignoring". The
responses where emphatic.

But yes, as usual the rest of Tim's comments were
important too.

> Tim wrote:
> 
> > We've talked about this before and last time the only thing I had
> > heartburn with was "always releasable" trunk as it implies an "Always
> > Branch" system and I think that's overly rigid and runs counter to our
> > community goals.  A reasonably stable trunk is a good goal.  Well
> > documented to the extent possible - if something is still under heavy
> > development then time shouldn't be wasted documenting yet (beyond that
> > which would allow other devs to check it out).  Heavy development in
> > the trunk?  Yes, if it's not causing the trunk to be unstable, then
> > yes.
> 
> One reason that I wrote the new proposal was the impression that we
> needed to reconsider our position in the light of last years
> developments.
> 
> > Heavy development in the trunk? Yes, if it's not causing the trunk
> > to be unstable, then yes.
> 
> In my view this rule has already lead to some type 1.5 and 2
> developers to opt out of using trunk in the last year. Something that
> I feel we cannot ignore.

Hmmm, apart from Dispatcher which was necessarily
changing (but still within the whiteboard) i cannot
see what would be a problem.

I agree, we need to address any such possibility.

> > As written, I think this would lead to fracturing in a couple ways.
> > 1) devs wouldn't check-out all the other branches and new stuff
> > would be sole-developed even more so than some things are now; 2)
> > bleeding-edge users won't get into checking out multiple branches to
> > checkout new functionality (i.e. no patches).
> 
> I (still) agree that this may be an effect. But in my view it is
> _already happening today_ and I'm merely trying to establish a system
> that will make sure that fractures are put back together at a certain
> point in the process.

Yes already happening.

We probably need to approach it from many angles, e.g.
encourage divisions not to happen, fix fractures,
ease the load of discussion, better email subjects, etc.

> > Encouraging such on the project would lead, in my opinion, to a
> > fractured community.  People naturally tend to prioritize email based
> > on the subject line, but supporting that through branch prefixes or
> > the like would lead to several "mini-projects".  Heck, people might
> > even set up email filters to only look at "their" branch -- not good. 
> > I think instead we need folks periodically reminding everyone to write
> > good archive/mailbox-friendly subject lines.
> 
> Perhaps I'm stuck too much in my old ways of thinking. But in all the
> organizations I have ever belonged to, it was an accepted and common
> practice to branch into working-groups that report back to the plenum.
> And I can't see why this should be wrong for Forrest.
> 
> Of course it would be ideal if we all had the time to follow all the
> discussions in detail but face it, the last year has shown that we
> don't, that people do filter and opt out already.

I know that. I struggle to keep some control of my
dev@cocoon inbox. But i do manage to do it.

Yes by ignoring some messages and by not committing
patches when i feel compelled. I do not do it because
of some sanctioned practice at Cocoon, but by deciding
for myself what to let pass.

Yes we all feel guilty for doing so, but that is
how it is. We cannot hope to cover everything,
but we can as a group.

> Back to David's message
> 
> > We already know that many people do not use a sensible
> > subject line for their email messages. So trying to
> > get them to add a prefix will be difficult too.
> 
> Agreed. And it will probably not always work. But wouldn't it be
> useful even if it only worked 80% of the time?

My main concern is potential fragmentation of
the community. So my answer is: i don't know.

> > Another issue is that discussions often fork into
> > some very important new general topic and people
> > do not change the subject line.
> 
> So one community effort - as I see it - would be to help and correct
> such mistakes.
> 
> > So important topics
> > will tend to be ignored.
> 
> Why are you saying 'will be'. This is ignoring the fact that it
> already happens to day. Just scan the list for Ross writing 'I really
> need to re-read this or that'. And I take that to be just the tip of
> the iceberg because Ross is very honest about this.

I meant "So important topics tend to be ignored."
i.e. apply to both today and the future. If people
are more careful about using sensible email subjects
then the problem becomes less.

> Doing a more detailed analysis of list interaction I could probably
> point out many many instances where people have completely ignored or
> misunderstood important parts of a thread.
> 
> And while this has many reasons (that we talked about recently), one
> big factor to me seems our attempts to cut corners by speed-reading,
> skipping, filtering (add your own ...) just to maintain this utopia
> of being able to do it all.

I agree. We cannot even expect to do it all.

> (Now you know why Germans are called the most pessimistic people in
> Europe :-))
> 
> > Another issue is that prefixes will enable people
> > to automatically filter mail out-of-sight. This would
> > lead to a fragmented community.
> 
> > If we just continually encourage the use of descriptive
> > email Subject lines and new ones for new topics,
> > then the main aim of your proposal is accomplished.
> > So there is no need to explictly talk about ignoring
> > some topics.
> 
> I disagree because I still wouldn't know when a new topic on
> structurer is so close to maturity that it makes sense for me to
> follow it.
> 
> > As Ross said in [A], one can peek into the first
> > posting of a thread, and then follow it or not.
> 
> Well, joining this project rather late, my experience is that this
> does not work at all. Quite often threads develop in all kinds of
> directions so you really have to read it all. My proposal won't fix
> that but it aims at improving this situation.
> 
> >>    Whiteboard and Peer Review Priciples
> >>      
> >>    It's important to point out that this way of using the whiteboard
> >>    is an attempt to deal with limited resources in a realistic way by
> >>    allowing committers with limited time, to 'savely ignore' the
> >>    discussions of early development stages and focus on paticipation
> >>    in later stages (or when they are explicitly asked for an opinion).
> 
> > I, for one, am not prepared to add the concept of
> > "safely ignore" to our project guidelines.
> 
> Perhaps we could call it "more safely ignore" or something else.
> 
> > It totally
> > goes against the Apache projects' PMC oversight principles.
> 
> Does it? If you think so I'd suggest that we do a reality check and
> test how many of our current committers are actually up to date on all
> current developments and discussion.

I am not saying that anyone can possibly keep up
with everything.

What i am saying is that to encourage people not to
look at discussion (and putting such terms in project
guidelines) goes against the principle of oversight.

If the PMC is big enough and active enough, then
with those many sets of eyes, we should be able
to cover it in general.

> I may be wrong but me guess is that I would not be the only one to
> fail miserably ... wdyt
> 
> And if I'm right, aren't we paying lip service to these rules by
> ignoring this?
> 
> > People can make up their own mind about what they follow
> > and what they don't follow. I don't see a need to explictly
> > sanction the ignoring of topics. KISS.
> 
> I disagree. By choosing to ignore structurer/dispatcher topics today I
> miss the important explanations of architecture and concept as well as
> not so important issues. So I don't see that choice.

I don't see why it will be any different
six months down the track. Those explanations
will be long gone and no-one will be able to
summarise them all at the time of moving a
particular plugin out of the whiteboard.

> >>   Minimum Requirements for internal release are:
> >>   (this may need further discussion)
> 
> > Also needs to gain "community". If the code is a
> > one-person affair then we will get into trouble.
> > I suppose that an actual Vote would express that,
> > because +1 also indicates a willingness to assist.
> 
> Thanks for adding that. Wouldn't hurt to put that in there because it
> probably isn't always the same. And it should be important to us.
> 
> > It should be stressed that stuff should move out of
> > the whiteboard as soon as possible (similar to how
> > branches should merge ASAP). Don't stay there until
> > it is complete.
> 
> If you mean as soon as the architecture is mature and stable I agree.
> Did you mean it this way?

Yes. However some of my reply got snipped.
I also said it is difficult to define when some
functionality is mature, stable, complete.
So i don't see how it can become a rule. As a vague
guideline it is okay.

> >> The outcome of the application can be one of the following (decided by
> >> a Lazy approval [1] after discussing the proposal):
> 
> > Using "Lazy approval" is intended for the normal
> > development process, i.e. just do it, and if someone
> > raises a alarm bell then we backtrack/fix/decide.
> > This encourages efficient development and is used
> > as often as possible.
> 
> > With your proposed change of process i do not see how
> > it will work.
> 
> > Using lazy approval makes it hard to get the beast
> > back into the whiteboard in the case of "Postponement"
> > or "Rejection".
> 
> Why is that?  And what would you recommend?

Oh, i think that i might have misunderstood
your proposal. Are you suggesting this? ...
Someone makes a Proposal to move something out of
the whiteboard, then we let it run for a while
(how long?), and if no-one says anything then
the normal "Lazy consensus" kicks in and they
just do it.

One problem i see though. The proposal takes
away the "oversight" and so normal ASF development
process is gone. So i wonder how Lazy consensus
can work.

> > There is already a concept about "deployed" plugins
> > and "released" plugins. I cannot see where this is
> > defined in our docs.
> 
> I take it this would be a new discussion?

I think so.

> > Also we haven't yet defined a process for official releases
> > of plugins as separate from the release of a whole package
> > of trunk (e.g. forrest-0.7) which includes all plugins
> > (including whiteboard) in their current state. Official
> > releases of any product requires a PMC vote.
> 
> This also would be a new topic?

Perhaps. It was intended as a lead-in to the next paragraph.

> > That leads to another problem with the current proposal.
> > When it comes time for a release of Forrest (e.g. 0.8)
> > then the PMC needs to vote on what is packaged with that
> > release. Currently that includes $FORREST_HOME/whiteboard/.
> > PMC members who have not been watching what is happening
> > would have difficulty with voting.

Once thing that i was trying to say was if we deliberately
encourage people not to look at stuff in the whiteboard
then we probably should not be releasing it. So far i have
been working on the assumption that we are looking at it.

> I have the same problem (needing to trust the judgement of my fellow
> committers) today.

That should not be a problem. That is how it works.
We trust the PMC to do their best. If we each do our
little bit, then the whole will succeed.

> > I wondered earlier in this thread if whiteboard should
> > be moved outside of our trunk. That might solve some
> > problems, but there are other factors against doing it.
> 
> I'd be just as happy to exclude whiteboard from official releases.
> What are the factors against doing it?

I already said them earlier in this current thread.
No time to find them now.

-David

Re: [Proposal] Refining our Development Process

[Ferdinand Soethe <fe...@apache.org>]

Thanks for your comments, David.



David Crossley wrote:

> Ferdinand Soethe wrote:
>> Since the debate on this seems to have died down, I have changed and
>> extended some parts of this paper to address the points raised by
>> David and Ross.
>> 
>> Now I would like to invite all for a last round of refinement before I
>> put this to a vote on Friday.

> When you do, please change the Subject line so that
> it explictly says "whiteboard".

I'd rather not do that because the changes go beyond the whiteboard
process and so a change in subject line might be misleading.

> There are many other
> aspects that have not been addressed by this proposal,
> so it does not apply to other development (non-whiteboard).

I thought "refining" reflected my attempt to improve on some parts
rather than re-writing it completely.
Perhaps we can find a better wording for this.

> http://forrest.apache.org/guidelines.html#development
> e.g. define how we add new functionality to plugins that
> have already emerged from the whiteboard and how to
> maintain a trunk that is "always usable".

> Another thing that i would like to say up front is that
> i find myself repeating stuff said in the last major
> discussions. I would encourage people to read this
> whole thread:
> [A]
>  [Proposal] Development process and a stable trunk
>  http://marc.theaimsgroup.com/?t=112504310100005

I'm certainly guilty of ignoring that thread when I wrote this
proposal. One of the reasons being that I got the impression that some
of the responses then were written on the bases of misunderstandings,
so that I wanted to get a fresh start.


> What is it that stops group 2 from operating?
> Why can't they use a particular SVN revision number
> of trunk that they know works?

[Sorry Thorsten, I know this will sound like Structurer-bashing
 again but it is such a good example. ]

The problem is that our 'stable trunk policy' _solved_
the problem of projects failing to work for broken code reasons

but it _did not solve_ the problem of projects breaking because of
repeated changes in the architecture of new features.

And it also did not solve the problem of people investing a lot of
time and effort in understanding an quickly evolving architectural
concept.

In other words I todays think that I'm glad that I didn't even
try and learn views or earlier structurer versions because I would
have wasted my time and others (by asking detailed questions like I do
ask now about the locationmaps).

But since this not always obvious at the time it was trying to
establish a process that more clearly marks a borderline between wild
creative hacking style development and a mature concept.



> As i said in the recent discussion, "whiteboard" is already
> in the "trunk". I think that you mean "move out of whiteboard".

Yes, sorry, I missed that sentence when I edited my paper.

> Tim, and all other respondents, raised some very
> important concerns in the previous proposal [A].

Sorry, I'm not very good at deciding which ones of those comments are
still valid for the current proposal. But since you pointed to Tims
concerns I'm happy to requote and address them here:

Tim wrote:

> We've talked about this before and last time the only thing I had
> heartburn with was "always releasable" trunk as it implies an "Always
> Branch" system and I think that's overly rigid and runs counter to our
> community goals.  A reasonably stable trunk is a good goal.  Well
> documented to the extent possible - if something is still under heavy
> development then time shouldn't be wasted documenting yet (beyond that
> which would allow other devs to check it out).  Heavy development in
> the trunk?  Yes, if it's not causing the trunk to be unstable, then
> yes.

One reason that I wrote the new proposal was the impression that we
needed to reconsider our position in the light of last years
developments.

> Heavy development in the trunk? Yes, if it's not causing the trunk
> to be unstable, then yes.

In my view this rule has already lead to some type 1.5 and 2
developers to opt out of using trunk in the last year. Something that
I feel we cannot ignore.

> As written, I think this would lead to fracturing in a couple ways.
> 1) devs wouldn't check-out all the other branches and new stuff
> would be sole-developed even more so than some things are now; 2)
> bleeding-edge users won't get into checking out multiple branches to
> checkout new functionality (i.e. no patches).

I (still) agree that this may be an effect. But in my view it is
_already happening today_ and I'm merely trying to establish a system
that will make sure that fractures are put back together at a certain
point in the process.

> Encouraging such on the project would lead, in my opinion, to a
> fractured community.  People naturally tend to prioritize email based
> on the subject line, but supporting that through branch prefixes or
> the like would lead to several "mini-projects".  Heck, people might
> even set up email filters to only look at "their" branch -- not good. 
> I think instead we need folks periodically reminding everyone to write
> good archive/mailbox-friendly subject lines.

Perhaps I'm stuck too much in my old ways of thinking. But in all the
organizations I have ever belonged to, it was an accepted and common
practice to branch into working-groups that report back to the plenum.
And I can't see why this should be wrong for Forrest.

Of course it would be ideal if we all had the time to follow all the
discussions in detail but face it, the last year has shown that we
don't, that people do filter and opt out already.

Back to David's message

> We already know that many people do not use a sensible
> subject line for their email messages. So trying to
> get them to add a prefix will be difficult too.

Agreed. And it will probably not always work. But wouldn't it be
useful even if it only worked 80% of the time?

> Another issue is that discussions often fork into
> some very important new general topic and people
> do not change the subject line.

So one community effort - as I see it - would be to help and correct
such mistakes.

> So important topics
> will tend to be ignored.

Why are you saying 'will be'. This is ignoring the fact that it
already happens to day. Just scan the list for Ross writing 'I really
need to re-read this or that'. And I take that to be just the tip of
the iceberg because Ross is very honest about this.

Doing a more detailed analysis of list interaction I could probably
point out many many instances where people have completely ignored or
misunderstood important parts of a thread.

And while this has many reasons (that we talked about recently), one
big factor to me seems our attempts to cut corners by speed-reading,
skipping, filtering (add your own ...) just to maintain this utopia
of being able to do it all.

(Now you know why Germans are called the most pessimistic people in
Europe :-))

> Another issue is that prefixes will enable people
> to automatically filter mail out-of-sight. This would
> lead to a fragmented community.

> If we just continually encourage the use of descriptive
> email Subject lines and new ones for new topics,
> then the main aim of your proposal is accomplished.
> So there is no need to explictly talk about ignoring
> some topics.

I disagree because I still wouldn't know when a new topic on
structurer is so close to maturity that it makes sense for me to
follow it.

> As Ross said in [A], one can peek into the first
> posting of a thread, and then follow it or not.

Well, joining this project rather late, my experience is that this
does not work at all. Quite often threads develop in all kinds of
directions so you really have to read it all. My proposal won't fix
that but it aims at improving this situation.

>>    Whiteboard and Peer Review Priciples
>>      
>>    It's important to point out that this way of using the whiteboard
>>    is an attempt to deal with limited resources in a realistic way by
>>    allowing committers with limited time, to 'savely ignore' the
>>    discussions of early development stages and focus on paticipation
>>    in later stages (or when they are explicitly asked for an opinion).

> I, for one, am not prepared to add the concept of
> "safely ignore" to our project guidelines.

Perhaps we could call it "more safely ignore" or something else.

> It totally
> goes against the Apache projects' PMC oversight principles.

Does it? If you think so I'd suggest that we do a reality check and
test how many of our current committers are actually up to date on all
current developments and discussion.

I may be wrong but me guess is that I would not be the only one to
fail miserably ... wdyt

And if I'm right, aren't we paying lip service to these rules by
ignoring this?

> People can make up their own mind about what they follow
> and what they don't follow. I don't see a need to explictly
> sanction the ignoring of topics. KISS.

I disagree. By choosing to ignore structurer/dispatcher topics today I
miss the important explanations of architecture and concept as well as
not so important issues. So I don't see that choice.

>>   Minimum Requirements for internal release are:
>>   (this may need further discussion)

> Also needs to gain "community". If the code is a
> one-person affair then we will get into trouble.
> I suppose that an actual Vote would express that,
> because +1 also indicates a willingness to assist.

Thanks for adding that. Wouldn't hurt to put that in there because it
probably isn't always the same. And it should be important to us.

> It should be stressed that stuff should move out of
> the whiteboard as soon as possible (similar to how
> branches should merge ASAP). Don't stay there until
> it is complete.

If you mean as soon as the architecture is mature and stable I agree.
Did you mean it this way?

>> The outcome of the application can be one of the following (decided by
>> a Lazy approval [1] after discussing the proposal):

> Using "Lazy approval" is intended for the normal
> development process, i.e. just do it, and if someone
> raises a alarm bell then we backtrack/fix/decide.
> This encourages efficient development and is used
> as often as possible.

> With your proposed change of process i do not see how
> it will work.


> Using lazy approval makes it hard to get the beast
> back into the whiteboard in the case of "Postponement"
> or "Rejection".

Why is that? And what would you recommend?


> There is already a concept about "deployed" plugins
> and "released" plugins. I cannot see where this is
> defined in our docs.

I take it this would be a new discussion?

> Also we haven't yet defined a process for official releases
> of plugins as separate from the release of a whole package
> of trunk (e.g. forrest-0.7) which includes all plugins
> (including whiteboard) in their current state. Official
> releases of any product requires a PMC vote.

This also would be a new topic?

> That leads to another problem with the current proposal.
> When it comes time for a release of Forrest (e.g. 0.8)
> then the PMC needs to vote on what is packaged with that
> release. Currently that includes $FORREST_HOME/whiteboard/.
> PMC members who have not been watching what is happening
> would have difficulty with voting.

I have the same problem (needing to trust the judgement of my fellow
committers) today.


> I wondered earlier in this thread if whiteboard should
> be moved outside of our trunk. That might solve some
> problems, but there are other factors against doing it.

I'd be just as happy to exclude whiteboard from official releases.
What are the factors against doing it?

--
Ferdinand Soethe

Re: [Proposal] Refining our Development Process

[David Crossley <cr...@apache.org>]

Ferdinand Soethe wrote:
> Since the debate on this seems to have died down, I have changed and
> extended some parts of this paper to address the points raised by
> David and Ross.
> 
> Now I would like to invite all for a last round of refinement before I
> put this to a vote on Friday.

When you do, please change the Subject line so that
it explictly says "whiteboard". There are many other
aspects that have not been addressed by this proposal,
so it does not apply to other development (non-whiteboard).
http://forrest.apache.org/guidelines.html#development
e.g. define how we add new functionality to plugins that
have already emerged from the whiteboard and how to
maintain a trunk that is "always usable".

Another thing that i would like to say up front is that
i find myself repeating stuff said in the last major
discussions. I would encourage people to read this
whole thread:
[A]
 [Proposal] Development process and a stable trunk
 http://marc.theaimsgroup.com/?t=112504310100005

> Different Approaches to Development
> -----------------------------------
> 
> There are two major approaches to software development represented in
> the this project at the moment:
> 
> - the evolutionary coding approach where concepts are developed while
>   coding and
> 
> - the classic approach of developing and discussing a concept in
>   theory that is then coded and refinded.
> 
> Both approaches may have certain merits that I'm not about to
> discuss here. Instead I like us acknowledge that both approaches
> can help us develop Forrest and that we need to create an environment
> where they can co-exist with each other and the needs of people who
> are using Forrest.
> 
> 
> Different Users
> ---------------
> 
> >From a user's perspective. I can see at
> least three different approaches:
> 
> 1. Very Early Adopters (VEA) will be happy to always work with the latest
>    unreleased version to try all the new features before
>    they are even released.
> 
>    They have the skills to fix things if need to and accept the high
>    cost of maintaining a site as a price for always having
>    access to the latest features for themselves (or their clients).
> 
> 1.5 Very technical people who are able to work on experimental
>     parts of the code base where that code adds benefit to their project.
>     However, they may not want to work with other experimental parts of the
>     code base.
> 
> 2. A second group, let's call them Early adopters (EA), is also
>    prepared to try and use new features in their projects, but for some
>    reason cannot run sites with a bleeding edge version.
> 
>    Typical reasons for this are:
> 
>    - Mission critical sites
>    - clients that don't accept downtime
>    - a limited budget to migrate projects along the stages of
>      bleeding edge development or
>    - lack of skills on the part of the EA,
>      so that quick fixes of problems are not always possible.
> 
> 3. A third group are users of released code (URC). For what ever
>    reasons they are happy to use a released and documented version and
>    have neither skill nor urge to use anything else.
> 
> As far as I can tell, we are currently doing ok for groups 1 and 3,
> while group 2 is quite often unable to work with trunk in their
> projects. As a result we are loosing valuable input and participation.

What is it that stops group 2 from operating?
Why can't they use a particular SVN revision number
of trunk that they know works?

> Different ways of Participation
> -------------------------------
> 
> Fortunately we are a community with very different backgrounds, which
> are often reflected in the different development approaches
> described earlier.
> 
> While some of us develop concepts through coding and re-factoring
> code, others prefer to approach a question from an architects point of view.
> 
> I think that it is import to understand that this difference is not
> always a matter of choice. Some people (like me) are simply to slow to
> read and understand code to join the first camp while others lack the
> patience (or ability) to think in concepts only.
> 
> We should design a development process that will allow both to
> follow developments, to participate and contribute.
> 
> How to achieve all that?
> ------------------------
> 
> My idea is actually quite simple and it basically extends the
> white-board concept already in place.
> 
> I'll try to explain it in a few headlines:
> 
> 1. Freedom of choice
> 
>   If a group of people get the itch to develop something for Forrest it
>   should be up to them to choose the development method that works best
>   for them.
> 
> 2. Whiteboard
> 
>    Any new feature development is in a whiteboard plugin, or if this is
>    not possible in a branch. It will not be integrated in trunk until
>    it is internally released (will define that in a moment).

As i said in the recent discussion, "whiteboard" is already
in the "trunk". I think that you mean "move out of whiteboard".
See other comments at the end of this email.

>    Being in whiteboard means:
> 
>    - people are completely free in how they approach development, if and
>      how they document it and how often they change their concepts.
> 
>    - they are encouraged to develop conceptual documentation early and
>      discuss it with the whole of the project (to make sure everybody
>      agrees with their architectural views) but there is no obligation to
>      do so.
> 
>    - up to the point of internal release committers and other developers
>      are welcome to but not expected to follow discussions on such a
>      development or to involve themselves in the development process.
>
>    - clear indicators (like '[WB]') in the subject line will help to
>      tell group internal threads from normal project-wide discussion so
>      that people can safely ignore group internal mails if they chose to
>      do so.

Tim, and all other respondents, raised some very
important concerns in the previous proposal [A].

We already know that many people do not use a sensible
subject line for their email messages. So trying to
get them to add a prefix will be difficult too.

Another issue is that discussions often fork into
some very important new general topic and people
do not change the subject line. So important topics
will tend to be ignored.

Another issue is that prefixes will enable people
to automatically filter mail out-of-sight. This would
lead to a fragmented community.

If we just continually encourage the use of descriptive
email Subject lines and new ones for new topics,
then the main aim of your proposal is accomplished.
So there is no need to explictly talk about ignoring
some topics.

As Ross said in [A], one can peek into the first
posting of a thread, and then follow it or not.

>    Whiteboard and Peer Review Priciples
>      
>    It's important to point out that this way of using the whiteboard
>    is an attempt to deal with limited resources in a realistic way by
>    allowing committers with limited time, to 'savely ignore' the
>    discussions of early development stages and focus on paticipation
>    in later stages (or when they are explicitly asked for an opinion).

I, for one, am not prepared to add the concept of
"safely ignore" to our project guidelines. It totally
goes against the Apache projects' PMC oversight principles.

People can make up their own mind about what they follow
and what they don't follow. I don't see a need to explictly
sanction the ignoring of topics. KISS.

>    It does _not mean_ that all committers can or should ignore what's
>    happening in the whiteboard. If time permits, peer review of
>    whiteboard development is as desirable (and sometimes even more
>    useful) as in later stages. Also, like today, peer reviewing
>    whiteboard developments or discussions is not tied to an
>    active involvelement in a feature development.
> 
>    Accepting that time is a limiting factor and people will (have to)
>    opt out of peer reviewing certain whiteboard developments (as they
>    today opt out of some discussion), it is important to notice the
>    increased responsibilities of those committers participating in
>    a whiteboard development (especially those committing code to the
>    whiteboard):
>
>    - Make sure that code you are committing is license compatible. If
>      in doubt raise this concern to a project wide discussion.
> 
>    - Encourage participants to document their plans and ideas in well
>      written, concise summaries and invite early project-wide feedback on it.
> 
>    - Pay attention to such papers from other whiteboard projects
>      and make sure that interface and compatibility issues are
>      addressed early on.
> 
>    In addition to all these safeguards documentation templates for
>    whiteboard development should carry full information about the
>    meaning of whiteboard status in Forrest.
> 
> 3: Internal release
> 
>   When the group is satisfied with architecture, features and
>   performance of their development they propose an 'internal release' to
>   integrate their development into trunk (or move a plugIn out of the
>   whiteboard).
> 
>   Minimum Requirements for internal release are:
>   (this may need further discussion)

Also needs to gain "community". If the code is a
one-person affair then we will get into trouble.
I suppose that an actual Vote would express that,
because +1 also indicates a willingness to assist.

>   - A well documented and stable architectural concept that the group is
>     prepared to defend in a project wide discussion.
> 
>   - Stable code that is mature enough not to break trunk
> 
>   - Basic low level practical documentation
>     = how to install
>     = what will break / what needs to be done to migrate
>     = how do I use its features
> 
>   - A suite of tests. For plugins that are well documented we have tests
>     built into the documentation. That is if the docs have examples of all
>     features, then doing "ant test" in the plugin directory will run the
>     tests accordingly.    
>     
>   - A preparedness to answer silly questions from people who try to
>     flesh out documentation in the next stage.

It should be stressed that stuff should move out of
the whiteboard as soon as possible (similar to how
branches should merge ASAP). Don't stay there until
it is complete.

What level of functionality is required? Just basic
i presume.

> The outcome of the application can be one of the following (decided by
> a Lazy approval [1] after discussing the proposal):

Using "Lazy approval" is intended for the normal
development process, i.e. just do it, and if someone
raises a alarm bell then we backtrack/fix/decide.
This encourages efficient development and is used
as often as possible.

With your proposed change of process i do not see how
it will work.

Using lazy approval makes it hard to get the beast
back into the whiteboard in the case of "Postponement"
or "Rejection".

>   - Acceptance
> 
>     The development is sound and complete and is considered to be a
>     useful extension of Forrest that we are happy to support as part
>     of Forrest. It will then be scheduled for the next release (see
>     4. below).
> 
>   - Postponement
> 
>     The development is either incomplete or has other deficiencies
>     (including poor design, performance etc.)
> 
>     It remains in whiteboard for improvement and can once again be
>     proposed for internal release later on.
> 
>   - Rejection because the proposal violates basic design concepts, adds
>     blinking text or is deemed useless by the project (add other
>     reasons).
> 
>     It may remain in whiteboard for everybody to use or give it another
>     try.
> 
> 
> 
> 4. External Release
> 
>   Developments accepted in the internal release process should be
>   scheduled for release as soon as possible.
>
>   As a rule we should not wait to accumulate several features but aim
>   to release each feature on its own so that users can familiarize
>   themselves with and test each new feature as soon as possible.
> 
>   The idea is that integrating, testing and documenting an already
>   well developed new feature will allow a great number of people to
>   involve themselves (in integration, testing, documentation) and
>   could perhaps become a subject for a Forrest Friday with many
>   different tasks that can be worked on in parallel.
> 
>   I'd also like to see this develop as phases in the project year
>   where many committers work together on a common goal and get familiar
>   with each others work.
> 
> --
> Ferdinand Soethe
> 
> [1] http://forrest.apache.org/guidelines.html#approvals

There are some other things that need considering ...

There is already a concept about "deployed" plugins
and "released" plugins. I cannot see where this is
defined in our docs.

Also we haven't yet defined a process for official releases
of plugins as separate from the release of a whole package
of trunk (e.g. forrest-0.7) which includes all plugins
(including whiteboard) in their current state. Official
releases of any product requires a PMC vote.

That leads to another problem with the current proposal.
When it comes time for a release of Forrest (e.g. 0.8)
then the PMC needs to vote on what is packaged with that
release. Currently that includes $FORREST_HOME/whiteboard/.
PMC members who have not been watching what is happening
would have difficulty with voting.

I wondered earlier in this thread if whiteboard should
be moved outside of our trunk. That might solve some
problems, but there are other factors against doing it.

-David

Re: [Proposal] Refining our Development Process

[Ferdinand Soethe <fe...@apache.org>]

Since the debate on this seems to have died down, I have changed and
extended some parts of this paper to address the points raised by
David and Ross.

Now I would like to invite all for a last round of refinement before I
put this to a vote on Friday.

Thanks,
Ferdinand


Different Approaches to Development
-----------------------------------

There are two major approaches to software development represented in
the this project at the moment:

- the evolutionary coding approach where concepts are developed while
  coding and

- the classic approach of developing and discussing a concept in
  theory that is then coded and refinded.

Both approaches may have certain merits that I'm not about to
discuss here. Instead I like us acknowledge that both approaches
can help us develop Forrest and that we need to create an environment
where they can co-exist with each other and the needs of people who
are using Forrest.


Different Users
---------------

From a user's perspective. I can see at
least three different approaches:

1. Very Early Adopters (VEA) will be happy to always work with the latest
   unreleased version to try all the new features before
   they are even released.

   They have the skills to fix things if need to and accept the high
   cost of maintaining a site as a price for always having
   access to the latest features for themselves (or their clients).

1.5 Very technical people who are able to work on experimental
    parts of the code base where that code adds benefit to their project.
    However, they may not want to work with other experimental parts of the
    code base.

2. A second group, let's call them Early adopters (EA), is also
   prepared to try and use new features in their projects, but for some
   reason cannot run sites with a bleeding edge version.

   Typical reasons for this are:

   - Mission critical sites
   - clients that don't accept downtime
   - a limited budget to migrate projects along the stages of
     bleeding edge development or
   - lack of skills on the part of the EA,
     so that quick fixes of problems are not always possible.

3. A third group are users of released code (URC). For what ever
   reasons they are happy to use a released and documented version and
   have neither skill nor urge to use anything else.

As far as I can tell, we are currently doing ok for groups 1 and 3,
while group 2 is quite often unable to work with trunk in their
projects. As a result we are loosing valuable input and participation.

Different ways of Participation
-------------------------------

Fortunately we are a community with very different backgrounds, which
are often reflected in the different development approaches
described earlier.

While some of us develop concepts through coding and re-factoring
code, others prefer to approach a question from an architects point of view.

I think that it is import to understand that this difference is not
always a matter of choice. Some people (like me) are simply to slow to
read and understand code to join the first camp while others lack the
patience (or ability) to think in concepts only.

We should design a development process that will allow both to
follow developments, to participate and contribute.

How to achieve all that?
------------------------

My idea is actually quite simple and it basically extends the
white-board concept already in place.

I'll try to explain it in a few headlines:

1. Freedom of choice

  If a group of people get the itch to develop something for Forrest it
  should be up to them to choose the development method that works best
  for them.

2. Whiteboard

   Any new feature development is in a whiteboard plugin, or if this is
   not possible in a branch. It will not be integrated in trunk until
   it is internally released (will define that in a moment).

   Being in whiteboard means:

   - people are completely free in how they approach development, if and
     how they document it and how often they change their concepts.

   - they are encouraged to develop conceptual documentation early and
     discuss it with the whole of the project (to make sure everybody
     agrees with their architectural views) but there is no obligation to
     do so.

   - up to the point of internal release committers and other developers
     are welcome to but not expected to follow discussions on such a
     development or to involve themselves in the development process.

   - clear indicators (like '[WB]') in the subject line will help to
     tell group internal threads from normal project-wide discussion so
     that people can safely ignore group internal mails if they chose to
     do so.

   Whiteboard and Peer Review Priciples
     
   It's important to point out that this way of using the whiteboard
   is an attempt to deal with limited resources in a realistic way by
   allowing committers with limited time, to 'savely ignore' the
   discussions of early development stages and focus on paticipation
   in later stages (or when they are explicitly asked for an opinion).

   It does _not mean_ that all committers can or should ignore what's
   happening in the whiteboard. If time permits, peer review of
   whiteboard development is as desirable (and sometimes even more
   useful) as in later stages. Also, like today, peer reviewing
   whiteboard developments or discussions is not tied to an
   active involvelement in a feature development.

   Accepting that time is a limiting factor and people will (have to)
   opt out of peer reviewing certain whiteboard developments (as they
   today opt out of some discussion), it is important to notice the
   increased responsibilities of those committers participating in
   a whiteboard development (especially those committing code to the
   whiteboard):

   - Make sure that code you are committing is license compatible. If
     in doubt raise this concern to a project wide discussion.

   - Encourage participants to document their plans and ideas in well
     written, concise summaries and invite early project-wide feedback on it.

   - Pay attention to such papers from other whiteboard projects
     and make sure that interface and compatibility issues are
     addressed early on.

   In addition to all these safeguards documentation templates for
   whiteboard development should carry full information about the
   meaning of whiteboard status in Forrest.

3: Internal release

  When the group is satisfied with architecture, features and
  performance of their development they propose an 'internal release' to
  integrate their development into trunk (or move a plugIn out of the
  whiteboard).

  Minimum Requirements for internal release are:
  (this may need further discussion)


  - A well documented and stable architectural concept that the group is
    prepared to defend in a project wide discussion.

  - Stable code that is mature enough not to break trunk

  - Basic low level practical documentation
    = how to install
    = what will break / what needs to be done to migrate
    = how do I use its features

  - A suite of tests. For plugins that are well documented we have tests
    built into the documentation. That is if the docs have examples of all
    features, then doing "ant test" in the plugin directory will run the
    tests accordingly.    
    
  - A preparedness to answer silly questions from people who try to
    flesh out documentation in the next stage.

The outcome of the application can be one of the following (decided by
a Lazy approval [1] after discussing the proposal):

  - Acceptance

    The development is sound and complete and is considered to be a
    useful extension of Forrest that we are happy to support as part
    of Forrest. It will then be scheduled for the next release (see
    4. below).

  - Postponement

    The development is either incomplete or has other deficiencies
    (including poor design, performance etc.)

    It remains in whiteboard for improvement and can once again be
    proposed for internal release later on.

  - Rejection because the proposal violates basic design concepts, adds
    blinking text or is deemed useless by the project (add other
    reasons).

    It may remain in whiteboard for everybody to use or give it another
    try.



4. External Release

  Developments accepted in the internal release process should be
  scheduled for release as soon as possible.

  As a rule we should not wait to accumulate several features but aim
  to release each feature on its own so that users can familiarize
  themselves with and test each new feature as soon as possible.

  The idea is that integrating, testing and documenting an already
  well developed new feature will allow a great number of people to
  involve themselves (in integration, testing, documentation) and
  could perhaps become a subject for a Forrest Friday with many
  different tasks that can be worked on in parallel.

  I'd also like to see this develop as phases in the project year
  where many committers work together on a common goal and get familiar
  with each others work.

--
Ferdinand Soethe

[1] http://forrest.apache.org/guidelines.html#approvals