Re: Project participation and hackability (was: [VOTE] merge locationmap branch with trunk)

[Tim Williams <wi...@gmail.com>]

Since this is about project process, hopefully by my opinions I'm not
violating some unspoken pmc-only line here...

On 6/25/05, Nicola Ken Barozzi <ni...@apache.org> wrote:
> 
> The discussion that has sparked following the request to merge
> incomplete branches in the trunk, along with Tim's comments, gives me
> the opportunity to discuss about how we could manage code change.
> 
> The goal is to increase developer participation and involvment in the
> Forrest Project, by reducing the barrier to entry.

<related>
As one whose recently climbed that barrier, I'd suggest that the
largest barrier of entry to forrest development is it's own cocoon
foundations.  While patches, trunks, and branches are common, sitemaps
and other cocooon concepts take a lot of learning (== time
investment==barrier).  Of course, I could just be the slow one of the
crowd too.  I suspect I'm not the only one that looked at forrest and
had the initial question, "where's the code?"
</related>

...
> 
> Imagine that all cutting-edge users can use the trunk in production. A
> patch is just a couple of actions away. And after incorporation, the
> check is instantaneous, and on a *real* test, as it actually get used.

I may be very well be in a unique situation but I can't imagine many
folks are able to run a trunk on a live production machine?  What
about the CM baggage?  If I were a client of that *real* test, I think
I'd be concerned.  If the recent JRE-version related discussion around
release time are any indiciation, this is not necessarily unique.

> This can only work if the trunk is always *usable*, and not only
> _buildable_. This can make our trunk be really bug-free, as there would
> really be a lot of eyes looking at the code.

I said before that I do like a usable trunk (i.e. buildable + runnable
with no hoop jumping).

> I would thus propose that the trunk be always *releaseable* at all
> times, and that all new functionality that interferes with usage of
> prior features be developed on separate branches.

This, however, is quite a committment.  While I'm for a usable trunk,
extending that though to a "releasable trunk" is more committment than
I'd want.  Maybe I'm reading too much into it but claiming that at any
given moment the trunk is [apache] release quality is bold.

> Furthremore, these branches should merge whenever possible between them
> in a single branch so that they can be coded together, and get merged
> with the trunk only when all developer-known bugs are fixed.

I understand that there will inevitably be dependencies between
branches but I don't care for merging branches into a single branch
(btw, wouldn't that ultimately become the defacto dev trunk?).  I hate
to keep bringing this up but in the current locationmap branch, we
have locationmap functionality==trunkable and
views==just-a-hair-shy-of-trunkable; the result is that it's difficult
to merge lcoationmap now even thought it's mature enough to be merged.
 Granted the hurdles we're facing there are trivial but it's a useful
example.  The point is that if more than one functional components
progress on independent paths towards maturity yet exist in the same
branch, then there would be a dificult time getting the most mature
functionality out the door independent of the others.  For those
inevitable dependencies, perhaps there's an idea of a branch of a
branch given that svn branch is but a copy?

> This will also make it easier for us to release often, and to learn to
> make smaller reversible changes rather than big ones that are hard to
> understand by other developers and users.
> 
> Let me know what you think.

I guess the summary is that this sounds like an "Always-Branch" system
as opposed to the more pragmatic "Branch-When-Needed system" and that
seems overly rigid for little return.  In other words, I doubt that a
lot of folks are able to run a trunk in a production environment and
so burdening yourselves with the overhead of maintaining a trunk in a
"releasable state" for what would amount to a handful of folks that
could use it (and likely already have svn loaded anyway) doesn't seem
worth it.

--tim

Re: Project participation and hackability

[Ross Gardler <rg...@apache.org>]

Tim Williams wrote:
> Since this is about project process, hopefully by my opinions I'm not
> violating some unspoken pmc-only line here...

We are a very open community. We do not discuss anything like this in 
private. The only stuff that goes on in private is votes for new 
committers and internal Apache stuff. Everyone should feel that their 
opinion is extremely valuable to the project.

I'm not replying to your points here because they are all mirrored by 
other people in other mails on this thread, so I am trying to avoid 
repeating myself.

Ross

Re: Project participation and hackability

[Addi <ad...@rocktreesky.com>]

Nicola Ken Barozzi wrote:

> Tim Williams wrote:
>
>> ....
>>
>>> Imagine that all cutting-edge users can use the trunk in production. A
>>> patch is just a couple of actions away. And after incorporation, the
>>> check is instantaneous, and on a *real* test, as it actually get used.
>>
>>
>> I may be very well be in a unique situation but I can't imagine many
>> folks are able to run a trunk on a live production machine?  What
>> about the CM baggage?  If I were a client of that *real* test, I think
>> I'd be concerned.  If the recent JRE-version related discussion around
>> release time are any indiciation, this is not necessarily unique.
>
>
> I have a small company with about 15 people that use the Intranet 
> site. Being also a developer, I have no problem in using the latest 
> code in "production". It's a small percentage of all usages, but still 
> relevant for us.

>
>>> This can only work if the trunk is always *usable*, and not only
>>> _buildable_. This can make our trunk be really bug-free, as there would
>>> really be a lot of eyes looking at the code.
>>
>>
>> I said before that I do like a usable trunk (i.e. buildable + runnable
>> with no hoop jumping).
>>
>>> I would thus propose that the trunk be always *releaseable* at all
>>> times, and that all new functionality that interferes with usage of
>>> prior features be developed on separate branches.
>>
>>
>> This, however, is quite a committment.  While I'm for a usable trunk,
>> extending that though to a "releasable trunk" is more committment than
>> I'd want.  Maybe I'm reading too much into it but claiming that at any
>> given moment the trunk is [apache] release quality is bold.
>
>
> Ok, probably the wording is too strong.
>
>>> Furthremore, these branches should merge whenever possible between them
>>> in a single branch so that they can be coded together, and get merged
>>> with the trunk only when all developer-known bugs are fixed.
>>
>>
>> I understand that there will inevitably be dependencies between
>> branches but I don't care for merging branches into a single branch
>> (btw, wouldn't that ultimately become the defacto dev trunk?).
>
>
> There should not be dependencies between branches. If there is, then 
> it should merge in a single branch.
>
>>> This will also make it easier for us to release often, and to learn to
>>> make smaller reversible changes rather than big ones that are hard to
>>> understand by other developers and users.
>>>
>>> Let me know what you think.
>>
>>
>> I guess the summary is that this sounds like an "Always-Branch" system
>> as opposed to the more pragmatic "Branch-When-Needed system" and that
>> seems overly rigid for little return.  In other words, I doubt that a
>> lot of folks are able to run a trunk in a production environment and
>> so burdening yourselves with the overhead of maintaining a trunk in a
>> "releasable state" for what would amount to a handful of folks that
>> could use it (and likely already have svn loaded anyway) doesn't seem
>> worth it.
>
>
> I am one of those, and I think that most Forrest developers are. If we 
> have at least a couple of others willing to use it, we're set.
>
> I have omitted another thing though, that I would like to release 
> Forrest *with* SVN stuff, so that patches can always be easy to make 
> and to send.
>
> As for the difficulty of maintaining a "releaseable" trunk, see my 
> next reply. Thanks for your mail, it helps :-)
>
I just wanted to pipe in here real quickly to add another data point to 
the discussion.  I am not a real dev, just a tweaker and poker, so most 
of this discussion is not really for me to get involved in but I do want 
to work with forrest and help as much as possible.  I do like the idea 
of a using trunk for production.  I am serving about 150 people on our 
intranet and wouldn't mind using it for our stuff.  We haven't 
implemented Forrest yet, but I plan to by October.  I already have our 
test system set up to run off trunk while I've been playing, so if trunk 
were always in a usable state, I don't have to change anything.

Addi

Re: Project participation and hackability

[Ross Gardler <rg...@apache.org>]

Nicola Ken Barozzi wrote:
> Ross Gardler wrote:
> ...
> 
>> Once we have an RC built we only allow bug fixes in trunk, new 
>> features are developer in a branch. 
> 
> 
> I wouldn't be that strong. If I'm adding a new feature to the trunk, but 
> this does not interfere with prior behaviour, there are no problems.
> It's just about keeping the trunk possibly bug-free, as far as the 
> developer is concerned.

The above was only when we were in the "moving to a release state" i.e. 
we had actually build and released a release candidate. During normal 
development I would agree on branches for major work only.

>> But this will prevent complete features being merged with trunk 
>> because incomplete features are brought into the branch. All that will 
>> happen there is that the branch will get into the state that trunk was 
>> in before the 0.7 release and we will split our devs between creating 
>> new features in the branch and maintaining trunk (not necessarily a 
>> bad thing, but do we have enough devs for this?)
> 
> 
> We should not get into a situation in which an incomplete feature merges 
> with a more complete branch. This happens when development is too long, 
> and people start using the branch as their trunk.

Right, I get it now. That is what David said with respect to the 
locationmap branch, views and the delay on the 0.7 release.

Ross

Re: Project participation and hackability

[Nicola Ken Barozzi <ni...@apache.org>]

Ross Gardler wrote:
...
> Once we have an RC built we only allow bug fixes in trunk, new features 
> are developer in a branch. 

I wouldn't be that strong. If I'm adding a new feature to the trunk, but 
this does not interfere with prior behaviour, there are no problems.
It's just about keeping the trunk possibly bug-free, as far as the 
developer is concerned.

If we use branches for all new features, we have the problem that nobody 
will check what is happening, like with views. We should really push for 
small reversible changes in the trunk, and use branches only when this 
is not possible (like in a new feature that needs core refactoring).

Smaller changes are more easily testable, reversible, and auditable by 
all. Big commits tend to go by without peer review.

> Subsequent RC's for that release should be 
> made every couple of weeks. Once we have a couple of weeks with no bug 
> fixes we release.
>
> Remember that plugins can have a separate release cycle. So only work on 
> core need move to a branch.

Yup, that's cool :-)

>>>> Furthremore, these branches should merge whenever possible between them
>>>> in a single branch so that they can be coded together, and get merged
>>>> with the trunk only when all developer-known bugs are fixed.
>>>
>>> I understand that there will inevitably be dependencies between
>>> branches but I don't care for merging branches into a single branch
>>> (btw, wouldn't that ultimately become the defacto dev trunk?).
>>
>> There should not be dependencies between branches. If there is, then 
>> it should merge in a single branch.
> 
> But this will prevent complete features being merged with trunk because 
> incomplete features are brought into the branch. All that will happen 
> there is that the branch will get into the state that trunk was in 
> before the 0.7 release and we will split our devs between creating new 
> features in the branch and maintaining trunk (not necessarily a bad 
> thing, but do we have enough devs for this?)

We should not get into a situation in which an incomplete feature merges 
with a more complete branch. This happens when development is too long, 
and people start using the branch as their trunk.

If this is not possible, it's the most incomplete feature that must 
incorporate the changes from the more complete one, not the opposite. 
This frees it from being stopped in merging with the trunk.

...
> In other words, I am +1 for an "always RC quality" or an "always beta 
> quality" (as Nicola said in another reply) trunk. I am also +1 for 
> "alpha" quality branches where necessary.

-- 
Nicola Ken Barozzi                   nicolaken@apache.org
             - verba volant, scripta manent -
    (discussions get forgotten, just code remains)
---------------------------------------------------------------------

Re: Project participation and hackability

[Ross Gardler <rg...@apache.org>]

Nicola Ken Barozzi wrote:
> Tim Williams wrote:


>> This, however, is quite a committment.  While I'm for a usable trunk,
>> extending that though to a "releasable trunk" is more committment than
>> I'd want.  Maybe I'm reading too much into it but claiming that at any
>> given moment the trunk is [apache] release quality is bold.
> 
> 
> Ok, probably the wording is too strong.

Yes, I would prefer to think of it as trunk is always "release 
candidate" releasable. Meaning us devs would use it in production, so 
adventurous users should be able to as well.

We do a pretty good job of fixing users problems as they emerge. Looking 
at our user lists we have a number of active devs there and an 
increasing number of users helping users. I think we are strong enough 
to support milestone releases.

This, I believe, would be as close as we can reasonably get to "always 
releasable". We should consider making this a part of our release 
process. That is, once someone says "I think trunk is ready for a new 
release" we should consider making an RC release immediately. We should 
make announcements about this on our mail lists, but not in the wider 
community.

Once we have an RC built we only allow bug fixes in trunk, new features 
are developer in a branch. Subsequent RC's for that release should be 
made every couple of weeks. Once we have a couple of weeks with no bug 
fixes we release.

Remember that plugins can have a separate release cycle. So only work on 
core need move to a branch.

>>> Furthremore, these branches should merge whenever possible between them
>>> in a single branch so that they can be coded together, and get merged
>>> with the trunk only when all developer-known bugs are fixed.
>>
>>
>> I understand that there will inevitably be dependencies between
>> branches but I don't care for merging branches into a single branch
>> (btw, wouldn't that ultimately become the defacto dev trunk?).
> 
> 
> There should not be dependencies between branches. If there is, then it 
> should merge in a single branch.

But this will prevent complete features being merged with trunk because 
incomplete features are brought into the branch. All that will happen 
there is that the branch will get into the state that trunk was in 
before the 0.7 release and we will split our devs between creating new 
features in the branch and maintaining trunk (not necessarily a bad 
thing, but do we have enough devs for this?)

>>> This will also make it easier for us to release often, and to learn to
>>> make smaller reversible changes rather than big ones that are hard to
>>> understand by other developers and users.
>>>
>>> Let me know what you think.
>>
>>
>> I guess the summary is that this sounds like an "Always-Branch" system
>> as opposed to the more pragmatic "Branch-When-Needed system" and that
>> seems overly rigid for little return.  In other words, I doubt that a
>> lot of folks are able to run a trunk in a production environment and
>> so burdening yourselves with the overhead of maintaining a trunk in a
>> "releasable state" for what would amount to a handful of folks that
>> could use it (and likely already have svn loaded anyway) doesn't seem
>> worth it.
> 
> 
> I am one of those, and I think that most Forrest developers are. If we 
> have at least a couple of others willing to use it, we're set.

A number of us devs appear to use trunk in production environments too 
(and at least one of us is using locationmap in a production as well). 
We can do so because we know the workarounds for the problems that 
exist. In addition, as I noted above, we are doing a superb community 
job of supporting our users. I think we can really capitalise on that at 
this point, just as long as trunk is not chock full of workarounds.

In other words, I am +1 for an "always RC quality" or an "always beta 
quality" (as Nicola said in another reply) trunk. I am also +1 for 
"alpha" quality branches where necessary.

Ross

Re: Project participation and hackability

[Nicola Ken Barozzi <ni...@apache.org>]

Tim Williams wrote:
> Since this is about project process, hopefully by my opinions I'm not
> violating some unspoken pmc-only line here...

Not at all! Everyone is very welcome to participate in the discussion, 
we need every help we can get. :-)

> On 6/25/05, Nicola Ken Barozzi <ni...@apache.org> wrote:
> 
...
> <related>
> As one whose recently climbed that barrier, I'd suggest that the
> largest barrier of entry to forrest development is it's own cocoon
> foundations.  While patches, trunks, and branches are common, sitemaps
> and other cocooon concepts take a lot of learning (== time
> investment==barrier).  Of course, I could just be the slow one of the
> crowd too.  I suspect I'm not the only one that looked at forrest and
> had the initial question, "where's the code?"
> </related>

;-)

I hope that making it all into smaller chunks will somewhat ease teh 
problems.

> ....
> 
>>Imagine that all cutting-edge users can use the trunk in production. A
>>patch is just a couple of actions away. And after incorporation, the
>>check is instantaneous, and on a *real* test, as it actually get used.
> 
> I may be very well be in a unique situation but I can't imagine many
> folks are able to run a trunk on a live production machine?  What
> about the CM baggage?  If I were a client of that *real* test, I think
> I'd be concerned.  If the recent JRE-version related discussion around
> release time are any indiciation, this is not necessarily unique.

I have a small company with about 15 people that use the Intranet site. 
Being also a developer, I have no problem in using the latest code in 
"production". It's a small percentage of all usages, but still relevant 
for us.

>>This can only work if the trunk is always *usable*, and not only
>>_buildable_. This can make our trunk be really bug-free, as there would
>>really be a lot of eyes looking at the code.
> 
> I said before that I do like a usable trunk (i.e. buildable + runnable
> with no hoop jumping).
> 
>>I would thus propose that the trunk be always *releaseable* at all
>>times, and that all new functionality that interferes with usage of
>>prior features be developed on separate branches.
> 
> This, however, is quite a committment.  While I'm for a usable trunk,
> extending that though to a "releasable trunk" is more committment than
> I'd want.  Maybe I'm reading too much into it but claiming that at any
> given moment the trunk is [apache] release quality is bold.

Ok, probably the wording is too strong.

>>Furthremore, these branches should merge whenever possible between them
>>in a single branch so that they can be coded together, and get merged
>>with the trunk only when all developer-known bugs are fixed.
> 
> I understand that there will inevitably be dependencies between
> branches but I don't care for merging branches into a single branch
> (btw, wouldn't that ultimately become the defacto dev trunk?).

There should not be dependencies between branches. If there is, then it 
should merge in a single branch.

>>This will also make it easier for us to release often, and to learn to
>>make smaller reversible changes rather than big ones that are hard to
>>understand by other developers and users.
>>
>>Let me know what you think.
> 
> I guess the summary is that this sounds like an "Always-Branch" system
> as opposed to the more pragmatic "Branch-When-Needed system" and that
> seems overly rigid for little return.  In other words, I doubt that a
> lot of folks are able to run a trunk in a production environment and
> so burdening yourselves with the overhead of maintaining a trunk in a
> "releasable state" for what would amount to a handful of folks that
> could use it (and likely already have svn loaded anyway) doesn't seem
> worth it.

I am one of those, and I think that most Forrest developers are. If we 
have at least a couple of others willing to use it, we're set.

I have omitted another thing though, that I would like to release 
Forrest *with* SVN stuff, so that patches can always be easy to make and 
to send.

As for the difficulty of maintaining a "releaseable" trunk, see my next 
reply. Thanks for your mail, it helps :-)

-- 
Nicola Ken Barozzi                   nicolaken@apache.org
             - verba volant, scripta manent -
    (discussions get forgotten, just code remains)
---------------------------------------------------------------------