Testing releases (was: AJAX samples broken)

[Bertrand Delacretaz <bd...@apache.org>]

Le 11 avr. 06 à 17:55, Jean-Baptiste Quenot a écrit :

> * Jean-Baptiste Quenot:
>
>> So   the  samples   need  to   be  adjusted   to  use   the  old
>> JXTemplateGenerator
>
> This is  why we  should vote for  a *release*  preferrably already
> deployed on  the zone and  not just for  the current state  of the
> branch.  Last-minute changes can have nasty side-effects...

I see your point, but even without changing our current procedures,  
proper testing of the release would have shown the bug.

I regret not voting -1 on the release last week instead of -0, as I  
didn't have time to test it and IIRC we had no precise testing  
reports, just some folks saying "seems to work for me".

We should have the courage to delay a release if our resources don't  
allow us to test it properly, and maybe work with temporary branches  
for releases, to avoid a too long code freeze.

If there is a Summer of Code this year, I think we should have a  
project to complete our automated tests - in this case, just testing  
that the Ajax sample page comes up correctly at the HTML level would  
have shown the problem.

Hmm..maybe two projects actually: one for plain HTML tests and one  
for automated tests of Ajax stuff.

-Bertrand

Re: Testing releases

[Carsten Ziegeler <cz...@apache.org>]

Bertrand Delacretaz wrote:
> I see your point, but even without changing our current procedures,  
> proper testing of the release would have shown the bug.
Yes, I think we did a poor release this time and I feel guilty of doing
the release without testing it like I did with the previous releases.
And I totally forgot to get feedback from the users about the current
release like we did in previous releases as well.
We should take care that we do not forget this with the next release.

> 
> I regret not voting -1 on the release last week instead of -0, as I  
> didn't have time to test it and IIRC we had no precise testing  
> reports, just some folks saying "seems to work for me".
Same here.

> 
> We should have the courage to delay a release if our resources don't  
> allow us to test it properly, and maybe work with temporary branches  
> for releases, to avoid a too long code freeze.
Hmm, yes, but it seems that the interest in a release has decreased for
this release and I fear no matter how long we delay and wait for
resources it would be very hard to get real feedback *and* work on the
feedback by fixing things. But we should try it anyway.

Carsten
-- 
Carsten Ziegeler - Open Source Group, S&N AG
http://www.s-und-n.de
http://www.osoco.org/weblogs/rael/