Change Management Suggestion (was: EmoticonsFilter in 2.6.0)

[Terry Steichen <te...@net-frame.com>]

Janne,

I don't think the problem is that you weren't "loud" enough.

Because I'm testing a fully operational JSPWiki-based system, I have 
been running the 'stable version' (2.4.xx) as suggested.  The neat new 
stuff in the 2.5.xx releases just took too much time to figure out, and 
caused too much instability as the bugs and feature parameters were 
being worked through.

In the future, when preparing a new release such as 2.6.0, we should 
probably emphasize the need to test the pending new release against the 
most recent *stable* version (ie, 2.4.x in this case), as well as versus 
the *cutting edge* version (2.5.xx). 

In this regard, do you have any idea of how many (if any) 2.6.0 testers 
so far have used code customized to 2.4.xx (templates, filters, plugins, 
etc.) versus 2.5.xx stuff? 

 From a personal standpoint, this transition (to 2.6.0) is making me 
increasingly nervous, because I don't know how to anticipate which of 
the (many) 2.5.xx changes might be incompatible with 2.4.xx.  In 
hindsight, I probably should have jumped in with testing of my own, but 
I had thought the changes affecting APIs and such were being normalized 
both ways between 2.4.x and 2.5.x.  Maybe that wasn't realistic, but 
that was my (obviously naive) belief.  Or, maybe the stable version was 
moved up to a more recent 2.4 release, and I missed it?

Terry

Janne Jalkanen wrote:
>> This seems like a pretty important change (that filters no longer work), 
>> yet this is the first I've heard of it.  I make extensive use of 
>> filters, and that means that I'll have to wait a bit longer before I 
>> really scrub through 2.6.0.  (Hope there aren't any more surprises like 
>> this.)
>>     
>
> This modification was made nine months ago.
>
> 2007-03-18  Janne Jalkanen  <Ja...@ecyrd.com>
>
>         * 2.5.30
>
>         * Fixed BugReferenceToRenamedPageNotCleared
>
>         * Fixed BugTableHeaderNotXHMTLCompliant
>
>         * Fixed BugInitializablePluginNotInitialized
>
>         * API Change: The signature of PageFilter.initialize() now
>         also includes the WikiEngine.  This was necessary, because otherwise
>         the initialize() -method was essentially useless for most use
>         cases.
>
> Sorry for not being louder about it, but yeah, it's been there and
> visible for everyone since last March.
>
> Also, JSPWiki 2.5.x has been in beta since last July (that is 6
> months).  This is exactly the kind of stuff I would like people to
> catch when I say "it's out in beta, please test".  There have been
> LOADS of changes under the hood, and we just can't remember to type up
> everything.  Anyone trying out their old filters on any of the betas
> would have caught this, muttered about it, and we would've remembered
> to put it on the WhatsNewIn2.6 list.
>
> It's there now.
>
> /Janne
>
>   

Re: Change Management Suggestion

[Janne Jalkanen <Ja...@ecyrd.com>]

> I hope I'm not coming across as negative or petty.  As explained  
> below, I think there's a real issue here that we would do well to  
> address now.

No worries, it's fine.  This is an issue that we will need to discuss  
anyway :)

> I didn't suggest that you have forced or should try to force  
> anything.  Perhaps the problem is me - that after all this time, I  
> don't properly understand what the adjective 'stable' really  
> implies.  Certainly, it's very desirable to be constantly evolving  
> new features; that's what the 2.5.xx series was for.  Yet, I think  
> we would all agree that we also need a dependable version that  
> people can use immediately, which is, I assume(d) is what 2.4.xx  
> was all about.

Yes, this is true.  Now 2.6 has taken it's place, and new development  
will go to 2.7.  (That is, once the SVN repo is up.  2.6.x branch  
should really be bug fixes/minor new functionality now.)

> The issue (and from reading your comment, I do think it is an  
> issue) is what can the user of a stable version expect from the  
> next stable (non-major) version?  Certainly, at a minimum, what we  
> should do is define what parts of the original stable version  
> 'break' when shifting to the new version (rather than have them pop  
> up after release about an issue which has been known for months).   
> Ideally, this would be accompanied by a smooth and simple upgrade  
> of some sort.  I mean, what's the point of having those who want to  
> immediately use the product be directed to a stable version which  
> is more or less guaranteed not to be compatible with the next  
> stable version?

A user?  They can expect to do an upgrade, and things will run  
smoothly.  We do provide instructions for that.

A developer?  They can expect to see things break on occasion, and a  
developer should keep track of the ChangeLog.

There is - at least from my side - very little motivation to keep the  
API backwards compatible in a manner which would be expected from  
commercial software.  For commercial software, there's a certain  
financial motivation to do so.  For open source - frankly, I think  
it's just too much effort.  Anyone is free to maintain an older,  
stable version, if they're depending on it.  That's the advantage of  
open source.

Having said that, we can somewhat keep the compatibility.  But at  
some point you simply *have* to break compatibility in order to keep  
the code base maintainable by *volunteer work force who just dips in  
and out on occasion and cannot afford to try to spend weeks and weeks  
to understand the codebase*.  Now, obviously, a better-designed  
external, or a developer API would help, but we don't have such a  
thing, nor do we even have any sort of a design for such a thing.  I  
do have some ideas (like making WikiEngine and WikiContext  
interfaces), though.

>>
> Excellent point.  So, at least as far as those particular plugins  
> are concerned, testing for 2.4.xx was done (at least by you).  Why  
> not include things like filters as well?  And why not make these  
> part of a standard set of compatibility tests?

Because we don't have standard compatibility tests.  We *do* have  
loads and loads of tests to make sure that regular wikiusers don't  
see a breakage.  But we do not have a standard compatibility test  
suite - we just have some plugins I've tried to run without  
modification.

If someone wants to take this task up, I'm all for it.  But frankly,  
I don't have the mentality to be a tester.  And even more frankly, I  
would probably just let the whole thing stagnate and die, 'cos I'm  
not interested in API testing.  I'm not saying we shouldn't do it -  
I'm just saying that I'm personally pretty much the wrong person to  
look up to in this case. :-/

So, we really need people who wish to test against existing plugins.

>> Jumping from 2.4 to 2.6 means that new stuff gets added, and
>> compatibility MAY be broken in minor ways (the PageFilter one is a
>> minor incompatibility, as filters can be fixed in about two  
>> minutes as
>> there is no behaviour change.
> Well, that's part of my question: are the incompatibilities all  
> minor, and if so, how do you know that (if there's no effort made  
> to map 2.4 into 2.6)?  Even more fundamentally, what are all the  
> known incompatibilities?  And, as for the two minute estimate, that  
> assumes you know about the incompatibility; tracing down the  
> bizzare behavior that might result can, as you know, consume a lot  
> more time than that.

If you are a developer, your compiler will tell you exactly what the  
problem is.  We added one extra parameter to the initialize() call,  
and added a destroy() call.  Your filter can use this information, if  
it needs to.

I did make the necessary changes to the built-in filters, but that  
was necessary to get the code base to compile.   It took about a minute.

> Yes, I think that's the bottom line.  While I do understand and  
> appreciate the reasoning between the 2.4xx/2.5xx parallel  
> development, I now wonder if that doesn't make compatibility  
> extraordinarily difficult.  Yet, we also want to continue feature  
> development.  I don't know how to best reconcile these objectives.

I would expect this thing to happen again with 2.8 and 3.0.  Looks  
like 3.0 is going to have a massive amount of changes, which is going  
to make e.g. patches really rather complicated (e.g. if the packages  
change), and I would expect that we will need to release some 2.8.x  
releases before 3.0 is stable.

Then again, if all compatibility is broken, we might not even need to  
keep track of it.

It looks like there's going to be a LGPL 2.6.1 with a bunch of bug  
fixes.  However, since we haven't yet branched off, and 2.8 is not  
scheduled to have any real API changes I don't think this is going to  
be a big issue.

The current 2.8 roadmap is here:

https://issues.apache.org/jira/browse/JSPWIKI? 
report=com.atlassian.jira.plugin.system.project:roadmap-panel

/Janne

Re: Change Management Suggestion

[Terry Steichen <te...@net-frame.com>]

I hope I'm not coming across as negative or petty.  As explained below, 
I think there's a real issue here that we would do well to address now.

Janne Jalkanen wrote:
> We have testers?  That's news to me...
>
> The thing is - we can't force anyone to test anything.  So it really
> all comes down to volunteers.  We can make policies and plans, but
> unless we have people executing them, they don't matter.
>   
I didn't suggest that you have forced or should try to force anything.  
Perhaps the problem is me - that after all this time, I don't properly 
understand what the adjective 'stable' really implies.  Certainly, it's 
very desirable to be constantly evolving new features; that's what the 
2.5.xx series was for.  Yet, I think we would all agree that we also 
need a dependable version that people can use immediately, which is, I 
assume(d) is what 2.4.xx was all about.

The issue (and from reading your comment, I do think it is an issue) is 
what can the user of a stable version expect from the next stable 
(non-major) version?  Certainly, at a minimum, what we should do is 
define what parts of the original stable version 'break' when shifting 
to the new version (rather than have them pop up after release about an 
issue which has been known for months).  Ideally, this would be 
accompanied by a smooth and simple upgrade of some sort.  I mean, what's 
the point of having those who want to immediately use the product be 
directed to a stable version which is more or less guaranteed not to be 
compatible with the next stable version?
> I have a set of plugins I do test against (and they are working nicely
> in 2.6 and 2.4).  I can't say anything about anyone else.
>   
Excellent point.  So, at least as far as those particular plugins are 
concerned, testing for 2.4.xx was done (at least by you).  Why not 
include things like filters as well?  And why not make these part of a 
standard set of compatibility tests?

> The thing is, most people don't care until they realize that there's a
> new stable out there.  Then they try it, and then they see things
> broken.  This could take months, if they have no problems with the
> existing stable.  And we have no way of knowing what they are doing
> with the software, so we can't even check for it.
>   
I do understand and appreciate the problem you mention above.  However, 
I think it's a reasonable thing to announce a new stable version on the 
discussion list and leave it up to users to upgrade or take their 
chances later.

> Jumping from 2.4 to 2.6 means that new stuff gets added, and
> compatibility MAY be broken in minor ways (the PageFilter one is a
> minor incompatibility, as filters can be fixed in about two minutes as
> there is no behaviour change.  
>   
Well, that's part of my question: are the incompatibilities all minor, 
and if so, how do you know that (if there's no effort made to map 2.4 
into 2.6)?  Even more fundamentally, what are all the known 
incompatibilities?  And, as for the two minute estimate, that assumes 
you know about the incompatibility; tracing down the bizzare behavior 
that might result can, as you know, consume a lot more time than that.

> Jumping from 2.x to 3.0 means that compatibility WILL be broken,
> unless by miraculous chance it works.
>   
I understand that in making a major version change, some 
incompatibilities are expected and reasonable.  (I do wonder, however, 
if we will continue parallel development paths as we did in 2.4.xx and 
2.5.xx? See following comment)

> Unfortunately, we don't have a systematic way of checking
> against compatibility.  This is something I would really like someone
> to pick up.
>   
Yes, I think that's the bottom line.  While I do understand and 
appreciate the reasoning between the 2.4xx/2.5xx parallel development, I 
now wonder if that doesn't make compatibility extraordinarily 
difficult.  Yet, we also want to continue feature development.  I don't 
know how to best reconcile these objectives.

Re: Change Management Suggestion (was: EmoticonsFilter in 2.6.0)

[Janne Jalkanen <ja...@iki.fi>]

> In this regard, do you have any idea of how many (if any) 2.6.0 testers 
> so far have used code customized to 2.4.xx (templates, filters, plugins, 
> etc.) versus 2.5.xx stuff? 

We have testers?  That's news to me...

The thing is - we can't force anyone to test anything.  So it really
all comes down to volunteers.  We can make policies and plans, but
unless we have people executing them, they don't matter.

I have a set of plugins I do test against (and they are working nicely
in 2.6 and 2.4).  I can't say anything about anyone else.

The thing is, most people don't care until they realize that there's a
new stable out there.  Then they try it, and then they see things
broken.  This could take months, if they have no problems with the
existing stable.  And we have no way of knowing what they are doing
with the software, so we can't even check for it.

> hindsight, I probably should have jumped in with testing of my own, but 
> I had thought the changes affecting APIs and such were being normalized 
> both ways between 2.4.x and 2.5.x.  Maybe that wasn't realistic, but 

No.  2.4.x should contain only modifications which retain
compatibility.  So any 2.4.x should be compatible with any other
2.4.x.

Jumping from 2.4 to 2.6 means that new stuff gets added, and
compatibility MAY be broken in minor ways (the PageFilter one is a
minor incompatibility, as filters can be fixed in about two minutes as
there is no behaviour change.  It's really just one modified
initialize() and one added destroy()).

Jumping from 2.x to 3.0 means that compatibility WILL be broken,
unless by miraculous chance it works.

http://www.jspwiki.org/wiki/VersioningProposal

The page is new, but the principles have been there from the
beginning.  Unfortunately, we don't have a systematic way of checking
against compatibility.  This is something I would really like someone
to pick up.

/Janne