Re: Change Management Suggestion

[Terry Steichen <te...@net-frame.com>]

I hope I'm not coming across as negative or petty.  As explained below, 
I think there's a real issue here that we would do well to address now.

Janne Jalkanen wrote:
> We have testers?  That's news to me...
>
> The thing is - we can't force anyone to test anything.  So it really
> all comes down to volunteers.  We can make policies and plans, but
> unless we have people executing them, they don't matter.
>   
I didn't suggest that you have forced or should try to force anything.  
Perhaps the problem is me - that after all this time, I don't properly 
understand what the adjective 'stable' really implies.  Certainly, it's 
very desirable to be constantly evolving new features; that's what the 
2.5.xx series was for.  Yet, I think we would all agree that we also 
need a dependable version that people can use immediately, which is, I 
assume(d) is what 2.4.xx was all about.

The issue (and from reading your comment, I do think it is an issue) is 
what can the user of a stable version expect from the next stable 
(non-major) version?  Certainly, at a minimum, what we should do is 
define what parts of the original stable version 'break' when shifting 
to the new version (rather than have them pop up after release about an 
issue which has been known for months).  Ideally, this would be 
accompanied by a smooth and simple upgrade of some sort.  I mean, what's 
the point of having those who want to immediately use the product be 
directed to a stable version which is more or less guaranteed not to be 
compatible with the next stable version?
> I have a set of plugins I do test against (and they are working nicely
> in 2.6 and 2.4).  I can't say anything about anyone else.
>   
Excellent point.  So, at least as far as those particular plugins are 
concerned, testing for 2.4.xx was done (at least by you).  Why not 
include things like filters as well?  And why not make these part of a 
standard set of compatibility tests?

> The thing is, most people don't care until they realize that there's a
> new stable out there.  Then they try it, and then they see things
> broken.  This could take months, if they have no problems with the
> existing stable.  And we have no way of knowing what they are doing
> with the software, so we can't even check for it.
>   
I do understand and appreciate the problem you mention above.  However, 
I think it's a reasonable thing to announce a new stable version on the 
discussion list and leave it up to users to upgrade or take their 
chances later.

> Jumping from 2.4 to 2.6 means that new stuff gets added, and
> compatibility MAY be broken in minor ways (the PageFilter one is a
> minor incompatibility, as filters can be fixed in about two minutes as
> there is no behaviour change.  
>   
Well, that's part of my question: are the incompatibilities all minor, 
and if so, how do you know that (if there's no effort made to map 2.4 
into 2.6)?  Even more fundamentally, what are all the known 
incompatibilities?  And, as for the two minute estimate, that assumes 
you know about the incompatibility; tracing down the bizzare behavior 
that might result can, as you know, consume a lot more time than that.

> Jumping from 2.x to 3.0 means that compatibility WILL be broken,
> unless by miraculous chance it works.
>   
I understand that in making a major version change, some 
incompatibilities are expected and reasonable.  (I do wonder, however, 
if we will continue parallel development paths as we did in 2.4.xx and 
2.5.xx? See following comment)

> Unfortunately, we don't have a systematic way of checking
> against compatibility.  This is something I would really like someone
> to pick up.
>   
Yes, I think that's the bottom line.  While I do understand and 
appreciate the reasoning between the 2.4xx/2.5xx parallel development, I 
now wonder if that doesn't make compatibility extraordinarily 
difficult.  Yet, we also want to continue feature development.  I don't 
know how to best reconcile these objectives.

Re: Change Management Suggestion

[Janne Jalkanen <Ja...@ecyrd.com>]

> I hope I'm not coming across as negative or petty.  As explained  
> below, I think there's a real issue here that we would do well to  
> address now.

No worries, it's fine.  This is an issue that we will need to discuss  
anyway :)

> I didn't suggest that you have forced or should try to force  
> anything.  Perhaps the problem is me - that after all this time, I  
> don't properly understand what the adjective 'stable' really  
> implies.  Certainly, it's very desirable to be constantly evolving  
> new features; that's what the 2.5.xx series was for.  Yet, I think  
> we would all agree that we also need a dependable version that  
> people can use immediately, which is, I assume(d) is what 2.4.xx  
> was all about.

Yes, this is true.  Now 2.6 has taken it's place, and new development  
will go to 2.7.  (That is, once the SVN repo is up.  2.6.x branch  
should really be bug fixes/minor new functionality now.)

> The issue (and from reading your comment, I do think it is an  
> issue) is what can the user of a stable version expect from the  
> next stable (non-major) version?  Certainly, at a minimum, what we  
> should do is define what parts of the original stable version  
> 'break' when shifting to the new version (rather than have them pop  
> up after release about an issue which has been known for months).   
> Ideally, this would be accompanied by a smooth and simple upgrade  
> of some sort.  I mean, what's the point of having those who want to  
> immediately use the product be directed to a stable version which  
> is more or less guaranteed not to be compatible with the next  
> stable version?

A user?  They can expect to do an upgrade, and things will run  
smoothly.  We do provide instructions for that.

A developer?  They can expect to see things break on occasion, and a  
developer should keep track of the ChangeLog.

There is - at least from my side - very little motivation to keep the  
API backwards compatible in a manner which would be expected from  
commercial software.  For commercial software, there's a certain  
financial motivation to do so.  For open source - frankly, I think  
it's just too much effort.  Anyone is free to maintain an older,  
stable version, if they're depending on it.  That's the advantage of  
open source.

Having said that, we can somewhat keep the compatibility.  But at  
some point you simply *have* to break compatibility in order to keep  
the code base maintainable by *volunteer work force who just dips in  
and out on occasion and cannot afford to try to spend weeks and weeks  
to understand the codebase*.  Now, obviously, a better-designed  
external, or a developer API would help, but we don't have such a  
thing, nor do we even have any sort of a design for such a thing.  I  
do have some ideas (like making WikiEngine and WikiContext  
interfaces), though.

>>
> Excellent point.  So, at least as far as those particular plugins  
> are concerned, testing for 2.4.xx was done (at least by you).  Why  
> not include things like filters as well?  And why not make these  
> part of a standard set of compatibility tests?

Because we don't have standard compatibility tests.  We *do* have  
loads and loads of tests to make sure that regular wikiusers don't  
see a breakage.  But we do not have a standard compatibility test  
suite - we just have some plugins I've tried to run without  
modification.

If someone wants to take this task up, I'm all for it.  But frankly,  
I don't have the mentality to be a tester.  And even more frankly, I  
would probably just let the whole thing stagnate and die, 'cos I'm  
not interested in API testing.  I'm not saying we shouldn't do it -  
I'm just saying that I'm personally pretty much the wrong person to  
look up to in this case. :-/

So, we really need people who wish to test against existing plugins.

>> Jumping from 2.4 to 2.6 means that new stuff gets added, and
>> compatibility MAY be broken in minor ways (the PageFilter one is a
>> minor incompatibility, as filters can be fixed in about two  
>> minutes as
>> there is no behaviour change.
> Well, that's part of my question: are the incompatibilities all  
> minor, and if so, how do you know that (if there's no effort made  
> to map 2.4 into 2.6)?  Even more fundamentally, what are all the  
> known incompatibilities?  And, as for the two minute estimate, that  
> assumes you know about the incompatibility; tracing down the  
> bizzare behavior that might result can, as you know, consume a lot  
> more time than that.

If you are a developer, your compiler will tell you exactly what the  
problem is.  We added one extra parameter to the initialize() call,  
and added a destroy() call.  Your filter can use this information, if  
it needs to.

I did make the necessary changes to the built-in filters, but that  
was necessary to get the code base to compile.   It took about a minute.

> Yes, I think that's the bottom line.  While I do understand and  
> appreciate the reasoning between the 2.4xx/2.5xx parallel  
> development, I now wonder if that doesn't make compatibility  
> extraordinarily difficult.  Yet, we also want to continue feature  
> development.  I don't know how to best reconcile these objectives.

I would expect this thing to happen again with 2.8 and 3.0.  Looks  
like 3.0 is going to have a massive amount of changes, which is going  
to make e.g. patches really rather complicated (e.g. if the packages  
change), and I would expect that we will need to release some 2.8.x  
releases before 3.0 is stable.

Then again, if all compatibility is broken, we might not even need to  
keep track of it.

It looks like there's going to be a LGPL 2.6.1 with a bunch of bug  
fixes.  However, since we haven't yet branched off, and 2.8 is not  
scheduled to have any real API changes I don't think this is going to  
be a big issue.

The current 2.8 roadmap is here:

https://issues.apache.org/jira/browse/JSPWIKI? 
report=com.atlassian.jira.plugin.system.project:roadmap-panel

/Janne