You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucy.apache.org by Marvin Humphrey <ma...@rectangular.com> on 2010/03/10 05:57:26 UTC

Stable releases under new namespaces

Greets,

As we've been working towards a release of the current KS trunk to CPAN as a
non-dev tarball, I've been continuing to ponder backwards compatibility
strategies, and I've come up with what I hope is an improved plan.

As expressed in the prior thread, I strongly believe that we should not just
up and break back compat on current KS "stable" branch users.  However,
backwards compatibility is a feature, and not everyone needs it to the same
degree.  In particular, active developers tend not to need strict API
compatibility -- it's not that hard to update your code so long as features
you're counting on aren't outright removed.

The mechanism by which we were going to protect stable branch users was to
move trunk to the namespace "KinoSearch2".  However, the downside of this
stratagem is that the people forced into awkward namespace adjustments are the
most active contributors.  Should Peter Karman, who just released
Search::Query::Dialect::KSx, rename that to use "KS2x" to indicate which
version of KS his extension is compatible with?  That's backwards -- we should
reward be rewarding our most valuable developers, not foisting inconveniences
on them while providing less active users with a convenience they may not
need.

One alternative is to release "stable" branches into new namespaces, while
keeping svn trunk named "KinoSearch" in perpetuity: "KinoSearch0" was under
consideration, but now I'm thinking we could fix a couple Unicode bugs,
require re-indexing and call it "KinoSearch1".  People for whom API backwards
compatibility was a requirement would find it there.  People who need the
latest features can simply use "KinoSearch".

One problem is that when we release into a new namespace, extensions such as
Peter's probably won't work with it.  However, extension developers will have
the option of doing basically the same thing as us: releasing stable branches
into a new namespace, updating those for bugfixes only, while continuing to
focus new feature development on current trunk.  There would also be a natural
process of attrition when extension developers drift towards inactivity --
they release a version of their extension that syncs with the last stable
release of the main library during their active period.

This isn't a perfect solution, but I think it's at least a satisfactory way to
resolve the current situation.  The "stable" branch has been highly stable for
a long time, but those users were still warned by that prominent ALPHA warning
at the top of the docs -- expecting them to make a one-time switch is
acceptable.

Given that it meets that threshold in my judgment... the technique of
releasing stable branches into new namespaces is an interesting experiment to
run.  We know what will happen if we just break back compat, but I'd like to
gather data on this alternative strategy to determine whether it makes sense
for Lucy.

Does anybody know if any other projects have ever used a similar release
strategy?

Marvin Humphrey


Re: [KinoSearch] Stable releases under new namespaces

Posted by Marvin Humphrey <ma...@rectangular.com>.
On Tue, Mar 16, 2010 at 04:30:19PM -0700, Marvin Humphrey wrote:
> I can easily imagine that some people might want to avoid trunk instability
> and develop with KinoSearch3 as a platform rather than KinoSearch, if they are
> sure it has all the functionality they need;

Indeed, I just headed over to the #mojomojo channel at irc.perl.org, and they
are looking forward to developing with KinoSearch3 once we spin it off.  They
won't be using "KinoSearch" -- which makes sense, given how MojoMojo gets
deployed.

Marvin Humphrey


Re: [KinoSearch] Stable releases under new namespaces

Posted by Marvin Humphrey <ma...@rectangular.com>.
On Tue, Mar 16, 2010 at 10:57:59PM +0000, Dermot wrote:
> I am not sure if this is a viable suggestion or one you've discounted
> already but a number of modules use the 'Compat' namespace for
> backward compatibility. 

It's a reasonable suggestion, and I hadn't considered it, but it doesn't seem
to me like it would offer us very much.

The CPAN modules which have "compat" in the name often mean different things by it.
For instance, Module::Build::Compat provides compatibility with MakeMaker,
while Apache2::compat actually provides a mod_perl 1.x shim that allows you to
migrate your code piecemeal.  Neither of those situations applies to us.

And unfortunately, it makes some module names which were already painfully
long even longer, e.g.
"KinoSearch3::Compat::Search::Collector::BitCollector".

I can easily imagine that some people might want to avoid trunk instability
and develop with KinoSearch3 as a platform rather than KinoSearch, if they are
sure it has all the functionality they need; I'd rather not impose ridiculous
package names on them.

Marvin Humphrey


Re: [KinoSearch] Stable releases under new namespaces

Posted by Dermot <pa...@googlemail.com>.
On 15 March 2010 01:28, Marvin Humphrey <ma...@rectangular.com> wrote:
> On Sat, Mar 13, 2010 at 02:29:49PM -0800, webmasters@ctosonline.org wrote:
>
>> On Mar 12, 2010, at 10:16 AM, Marvin Humphrey wrote:
>> >  KinoSearch1 1.00    <-- branches/ks1
>> >  KinoSearch3 3.00    <-- branches/ks3
>> >  KinoSearch  4.00    <-- branches/maint-4.x
>> >  KinoSearch  5.00_01 <-- trunk
>>
...
>> How would I write a KSx::  module that works with multiple KinoSearch
>> versions?
>
> I'd recommend against doing so.
>
> KSx modules should target the current "KinoSearch" release only -- they should
> not work KinoSearch1, KinoSearch3 or other stable forks.  If you want to
> support KinoSearch3, release a back-fork into KS3x.  If you get tired of
> perpetually modifying an extension to keep up with trunk, move it to the last
> stable branch and go inactive.
...t.
>
>> Would I have to  keep releasing a new KS3x:: or KS4x:: whenever there is a
>> new KinoSearch release?
>
> Whenever there is a major version increment, yes.  But the ongoing support
> burden should not be that great.
>
> It would be up to you whether you ever wanted to fix bugs on one of those
> forks, for example.  Unless it's something really serious, I wouldn't bother.
>
> Though there won't be anything stopping people from developing against a
> stable fork like KinoSearch3 from the get-go, these forks are mostly a way to
> insulate mature, immobile code bases from the changes on trunk.  I doubt we'll
> back-port features for them, just as we haven't back-ported improvements to
> stable-branch KinoSearch 0.1x over the last two years.  So there will be
> almost no churn, and thus very little potential for breakage due to changes in
> the main library.
>
>> Right now I’m having a maintenance nightmare, as two separate websites
>> are using different versions of KinoSearch that are no longer on CPAN.
>> And I haven’t had time to update the code for either of them.... And I
>> need to update my KSx modules, too. I’m just looking for a way to make
>> this all much easier.
>
> I understand and I want to help.  You haven't been very active lately, but
> historically you have made many valuable contributions.  I want to make sure
> that we don't abandon you, the same way I want to make sure we don't abandon
> MojoMojo or Socialtext.


I am not sure if this is a viable suggestion or one you've discounted
already but a number of modules use the 'Compat' namespace for
backward compatibility. So you might have a KinoSearch2::Compat that
allows sites to maintain backward compatibility by using that
namespace and new development can use KinoSearch2. Admittedly the KS
indexes may not be compatible but you do get to keep the whole
distribution together.

Thought I'd throw that in :)
Dp.

Re: Stable releases under new namespaces

Posted by Marvin Humphrey <ma...@rectangular.com>.
Hello, Father...

We can help you by releasing a KinoSearch3 stable fork.  If keeping your
extension modules and code bases up to date with KS trunk is no longer
workable for you, we can at least give you a stable platform to work with that
supports all the features you've been using.

What we can't do is mimic Perl 5 and provide you with perpetual backwards
compatibility under the namespace "KinoSearch".  KinoSearch/Lucy must continue
to progress, and that will sometimes mean breaking back compat.

On Sat, Mar 13, 2010 at 02:29:49PM -0800, webmasters@ctosonline.org wrote:

> On Mar 12, 2010, at 10:16 AM, Marvin Humphrey wrote:
> >  KinoSearch1 1.00    <-- branches/ks1
> >  KinoSearch3 3.00    <-- branches/ks3
> >  KinoSearch  4.00    <-- branches/maint-4.x
> >  KinoSearch  5.00_01 <-- trunk
> 
> One of the problems with this approach is that, if I want to use  
> KinoSearch 4.00 stable, I would have to start with ‘KinoSearch’ and  
> later migrate my code to ‘KinoSearch4’.

The plan in the quote above is an alternate which I also had problems with but
was exploring for the purpose of argument.  With multiple objections having
been lodged against it now, I think we can rule it out.

The original plan doesn't suffer from the defect you describe.

> Another problem with the all the multiple namespace suggestions so far  
> is that it makes it hard for KSx:: authors. 

In some sense, it's hard for KSx authors no matter what unless we adopt
perpetual backwards compat as a policy (which has its own downside, of
course).  You are having difficulties right now because the main library has
moved out from underneath you and you have multiple installs with slightly
different APIs to support.  This problem has arisen *without* us releasing
stable forks into new namespaces.

> How would I write a KSx::  module that works with multiple KinoSearch
> versions? 

I'd recommend against doing so.

KSx modules should target the current "KinoSearch" release only -- they should
not work KinoSearch1, KinoSearch3 or other stable forks.  If you want to
support KinoSearch3, release a back-fork into KS3x.  If you get tired of
perpetually modifying an extension to keep up with trunk, move it to the last
stable branch and go inactive.

Note that the alternative is for us to migrate trunk to the namespace
"KinoSearch2", in which case you'd have to fork your module into KS2x to work
with the *new* branch.  So keeping up with the main library while providing
back compat for older versions involves roughly the same amount of forking no
matter what.  

> Would I have to  keep releasing a new KS3x:: or KS4x:: whenever there is a
> new KinoSearch release?

Whenever there is a major version increment, yes.  But the ongoing support
burden should not be that great.

It would be up to you whether you ever wanted to fix bugs on one of those
forks, for example.  Unless it's something really serious, I wouldn't bother.

Though there won't be anything stopping people from developing against a
stable fork like KinoSearch3 from the get-go, these forks are mostly a way to
insulate mature, immobile code bases from the changes on trunk.  I doubt we'll
back-port features for them, just as we haven't back-ported improvements to
stable-branch KinoSearch 0.1x over the last two years.  So there will be
almost no churn, and thus very little potential for breakage due to changes in
the main library.

> Right now I’m having a maintenance nightmare, as two separate websites  
> are using different versions of KinoSearch that are no longer on CPAN.  
> And I haven’t had time to update the code for either of them.... And I  
> need to update my KSx modules, too. I’m just looking for a way to make  
> this all much easier.

I understand and I want to help.  You haven't been very active lately, but
historically you have made many valuable contributions.  I want to make sure
that we don't abandon you, the same way I want to make sure we don't abandon
MojoMojo or Socialtext.

To clear up the situation, all these code bases will need to be synchronized
on *some* version of KS.

You could bring them all up to the current KinoSearch trunk, then just avoid
upgrading your installed KinoSearch or KSx modules.  That's similar to what
you'd have to do if we simply broke back compat at major version increments.
(The fact that you have to resort to such techniques is a consequence of
Perl's dynamic loading mechanism, which as you know does not allow multiple
versions of the same library to co-exist.)

You could bring them all up to the current KinoSearch trunk, and keep them
up-to-date as the library evolves.  As you already are aware, this involves a
certain amount of ongoing work which active users accept, but is troublesome
for inactive users.

Or you could move them over to a stable fork, which will involve the least
amount of ongoing work once the transition is complete.  You won't get the
latest, greatest features anymore, but that may not matter -- the stable fork
may offer you all the features you need and serve as a sufficient development
platform for a long time to come.

I think this last option is your best one, and I'm willing to help by moving
towards a KinoSearch3 3.0 release.

Marvin Humphrey


Re: [KinoSearch] Stable releases under new namespaces

Posted by Marvin Humphrey <ma...@rectangular.com>.
On Sun, Mar 14, 2010 at 06:16:25PM -0700, webmasters@ctosonline.org wrote:
> I don’t understand all of what you are saying, but I had in mind  
> making KinoSearch an alias for KinoSearch4, and not vice versa.  
> Running multiple versions in the same process I don’t consider a  
> necessity. Does that change anything?

Theoretically less impossible, but still impossible. :(  And I do think that
the inability to use the modules side by side while aliased is a major fault.
With the separate namespaces, it's actually possible for libraries with
dependencies on different stable forks of KS to co-exist happily.  That's not
true if everybody's fighting over who gets to own "KinoSearch".

What you describe would be hard to pull off for a pure Perl distribution of
substantial size.  (Race conditions as to which module gets loaded first
determines what version of KS gets aliased, oi...)  Once you add in the
compiled C code, the consequences of failure, and the fact that the Clownfish
object model is already ambitious and totally unprepared for being abused in
such a way, it's just not practical.

Marvin Humphrey


Re: [KinoSearch] Stable releases under new namespaces

Posted by we...@ctosonline.org.
On Mar 14, 2010, at 5:21 PM, Marvin Humphrey wrote:

> On Sun, Mar 14, 2010 at 02:30:48PM -0500, Peter Karman wrote:
>> I like this approach. It's more work for KS developers, in terms of  
>> managing
>> api versions, but that's good work to do, since it requires making  
>> the api
>> changes very explicit.
>
> I understand that this sounds attractive in theory, but it's not  
> practical.
> Certainly having two versions run side by side won't ever work.  We  
> can't
> search-and-replace within compiled XS objects to substitute  
> "KinoSearch" for
> "KinoSearch3".  Does Query_Is_A(query, ORQUERY) pass, fail, or  
> segfault
> when "query" was created under KinoSearch3, but ORQUERY is a VTable  
> owned by
> KinoSearch4, aliased to KinoSearch?  What should factory methods  
> spit out, and
> how will they know?  How will type checking across the Perl/C border  
> work?

I don’t understand all of what you are saying, but I had in mind  
making KinoSearch an alias for KinoSearch4, and not vice versa.  
Running multiple versions in the same process I don’t consider a  
necessity. Does that change anything?

>
> Changing up behaviors using an api_version key can work within a  
> single
> object, using one of the following strategies:
>
>  * Store api_version as a member variable and switch up behavior  
> based on
>    its value.  That's what HTML::Parser does, I believe.
>  * Have the constructor dispatch based on api_version and return  
> objects
>    blessed into different classes.
>  * Store an API-specific object within the main object and dispatch  
> to that.
>
> However, KinoSearch is a highly performance-sensitive library with  
> hundreds of
> classes and lots of interaction between all the various parts.   
> Furthermore,
> it doesn't use hash-based method dispatch, but compile time binding  
> -- which
> is much faster, but also means that invoking an invalid method on an  
> object
> results in a memory error rather than a catchable exception.  At  
> best that's a
> segfault, at worst it's a security hole.
>
> Marvin Humphrey
>
>
> _______________________________________________
> kinosearch mailing list
> kinosearch@rectangular.com
> http://rectangular.com/cgi-bin/mailman/listinfo/kinosearch


Re: Stable releases under new namespaces

Posted by Peter Karman <pe...@peknet.com>.
Marvin Humphrey wrote on 3/14/10 7:21 PM:
> On Sun, Mar 14, 2010 at 02:30:48PM -0500, Peter Karman wrote:
>> I like this approach. It's more work for KS developers, in terms of managing
>> api versions, but that's good work to do, since it requires making the api
>> changes very explicit.
> 
> I understand that this sounds attractive in theory, but it's not practical.

Fair enough. I am a sucker for what sounds attractive. ;)


-- 
Peter Karman  .  http://peknet.com/  .  peter@peknet.com

Re: Stable releases under new namespaces

Posted by Marvin Humphrey <ma...@rectangular.com>.
On Sun, Mar 14, 2010 at 02:30:48PM -0500, Peter Karman wrote:
> I like this approach. It's more work for KS developers, in terms of managing
> api versions, but that's good work to do, since it requires making the api
> changes very explicit.

I understand that this sounds attractive in theory, but it's not practical.
Certainly having two versions run side by side won't ever work.  We can't
search-and-replace within compiled XS objects to substitute "KinoSearch" for
"KinoSearch3".  Does Query_Is_A(query, ORQUERY) pass, fail, or segfault
when "query" was created under KinoSearch3, but ORQUERY is a VTable owned by
KinoSearch4, aliased to KinoSearch?  What should factory methods spit out, and
how will they know?  How will type checking across the Perl/C border work?

Changing up behaviors using an api_version key can work within a single
object, using one of the following strategies:
  
   * Store api_version as a member variable and switch up behavior based on
     its value.  That's what HTML::Parser does, I believe.
   * Have the constructor dispatch based on api_version and return objects
     blessed into different classes.
   * Store an API-specific object within the main object and dispatch to that.

However, KinoSearch is a highly performance-sensitive library with hundreds of
classes and lots of interaction between all the various parts.  Furthermore,
it doesn't use hash-based method dispatch, but compile time binding -- which
is much faster, but also means that invoking an invalid method on an object
results in a memory error rather than a catchable exception.  At best that's a
segfault, at worst it's a security hole.

Marvin Humphrey


Re: Stable releases under new namespaces

Posted by Peter Karman <pe...@peknet.com>.
webmasters@ctosonline.org wrote on 3/13/10 4:29 PM:

> 
> Perhaps the ‘KinoSearch’ namespace could be a front end that delegates
> to a separate namespace. So ‘use KinoSearch 3;’ or ‘use KinoSearch api
> => 3’ would alias all the KinoSearch namespaces to KinoSearch3:: or
> KinoSearch::3:: for that process (or thread). Then a KSx module can call
> KinoSearch->api or api_version (perhaps this method could be on every
> object under KinoSearch::) to find out which version is being used. If a
> process needs to access multiple KinoSearch versions, it can use
> KinoSearch3 directly.
> 
> Using KinoSearch without specifying a version will default to the
> latest, with a warning. ‘api => "*"’, ‘api => 0’ or ‘api => "latest"’
> could be used to suppress the warning.
> 
> Now, for this to work seamlessly, all KinoSearch versions would have to
> be bundled along with the latest, which is a bit much. I think clear
> documentation that explains that KinoSearch2/3 needs to be installed
> separately (if it is an old version) is a sufficient compromise, as the
> ‘Can’t locate KinoSearch2.pm in @INC’ message explains exactly what’s
> wrong.

I like this approach. It's more work for KS developers, in terms of managing api
versions, but that's good work to do, since it requires making the api changes
very explicit.

Could put the api version in the schema.json file and provide clear error
messages when trying to read/write incompatible versions.

-- 
Peter Karman  .  http://peknet.com/  .  peter@peknet.com

Re: Stable releases under new namespaces

Posted by we...@ctosonline.org.
On Mar 12, 2010, at 10:16 AM, Marvin Humphrey wrote:
> I'd been thinking that "KinoSearch" would have a perpetually  
> unstable API, and
> that we'd abandon all pretense that it would ever be stable.   
> However, we
> could do as you say here and break API compatibility only at major  
> release
> points -- however, to avoid abandoning the earlier user base  
> completely, we'd
> spin off a renamed stable branch at each back-compat break.
>
> If we did that, CPAN would look like this right away:
>
>   KinoSearch1 1.00    <-- branches/ks1
>   KinoSearch  1.00    <-- identical to branches/ks1 except for  
> namespace
>   KinoSearch  3.00_01 <-- trunk
>
> We would then work to get trunk ready for a real version 3 release.   
> Once it
> was published, CPAN would look like this:
>
>   KinoSearch1 1.00    <-- branches/ks1
>   KinoSearch  3.00    <-- branches/maint-3.x
>   KinoSearch  4.00_01 <-- trunk
>
> Then, in a year or so[2] when we release KinoSearch 4, CPAN would  
> look like
> this:
>
>   KinoSearch1 1.00    <-- branches/ks1
>   KinoSearch3 3.00    <-- branches/ks3
>   KinoSearch  4.00    <-- branches/maint-4.x
>   KinoSearch  5.00_01 <-- trunk

One of the problems with this approach is that, if I want to use  
KinoSearch 4.00 stable, I would have to start with ‘KinoSearch’ and  
later migrate my code to ‘KinoSearch4’.

Another problem with the all the multiple namespace suggestions so far  
is that it makes it hard for KSx:: authors. How would I write a KSx::  
module that works with multiple KinoSearch versions? Would I have to  
keep releasing a new KS3x:: or KS4x:: whenever there is a new  
KinoSearch release?

I have a suggestion, which I think would make it much easier for  
script authors, and a little easier for module authors:

Perhaps the ‘KinoSearch’ namespace could be a front end that delegates  
to a separate namespace. So ‘use KinoSearch 3;’ or ‘use KinoSearch api  
=> 3’ would alias all the KinoSearch namespaces to KinoSearch3:: or  
KinoSearch::3:: for that process (or thread). Then a KSx module can  
call KinoSearch->api or api_version (perhaps this method could be on  
every object under KinoSearch::) to find out which version is being  
used. If a process needs to access multiple KinoSearch versions, it  
can use KinoSearch3 directly.

Using KinoSearch without specifying a version will default to the  
latest, with a warning. ‘api => "*"’, ‘api => 0’ or ‘api => "latest"’  
could be used to suppress the warning.

Now, for this to work seamlessly, all KinoSearch versions would have  
to be bundled along with the latest, which is a bit much. I think  
clear documentation that explains that KinoSearch2/3 needs to be  
installed separately (if it is an old version) is a sufficient  
compromise, as the ‘Can’t locate KinoSearch2.pm in @INC’ message  
explains exactly what’s wrong.


Right now I’m having a maintenance nightmare, as two separate websites  
are using different versions of KinoSearch that are no longer on CPAN.  
And I haven’t had time to update the code for either of them.... And I  
need to update my KSx modules, too. I’m just looking for a way to make  
this all much easier.

Father Chrysostomos


Re: Stable releases under new namespaces

Posted by Peter Karman <pe...@peknet.com>.
Marvin Humphrey wrote on 3/12/10 12:16 PM:

> Under the proposed policy, we privilege and reward our most active users and
> contributors.  However, we don't completely screw over people who were once
> active but have since become inactive.  They are required to make a one-time
> sweep through their code to use the spun-off stable branch.
> 
> This way, projects like MojoMojo and Socialtext -- which depend on KinoSearch
> but may not control exactly when Perl modules get upgraded on a machine
> running their software -- will be able to switch to a stable branch like
> KinoSearch1 and avoid breakage.

I agree on both those major points (reward activity, do no harm to legacy
users), and I guess I don't really have a strong opinion on the specific
implementation. Separate namespaces for stable, older releases seems like a
decent workaround for the limitations in how Perl/CPAN manage (or don't manage)
versioning.

-- 
Peter Karman  .  http://peknet.com/  .  peter@peknet.com

Re: Stable releases under new namespaces

Posted by Marvin Humphrey <ma...@rectangular.com>.
On Thu, Mar 11, 2010 at 03:35:07PM -0600, Peter Karman wrote:
> Marvin Humphrey wrote on 3/9/10 10:57 PM:
> > 
> > This isn't a perfect solution, but I think it's at least a satisfactory
> > way to resolve the current situation.  The "stable" branch has been highly
> > stable for a long time, but those users were still warned by that
> > prominent ALPHA warning at the top of the docs -- expecting them to make a
> > one-time switch is acceptable.
> 
> I've been delaying in my response to this because (a) it took me awhile to
> grok the idea, and (b) I've been trying to play through scenarios based on
> my desire to keep my KS extension modules happy.
> 
> I *think* I understand the idea. I.e., KinoSearch 0.1x becomes KinoSearch1,
> KinoSearch 0.3x becomes KinoSearch3, etc. Is that right?

That's a slight improvement on what had in mind.  :) 

I had been thinking that 0.3x would become KinoSearch2.  (Really 0.20_xx
should have been KinoSearch2, but it's long gone.)  However, I like the idea
that 0.3x corresponds to KinoSearch3.

So, here's what would happen right away:

  CPAN now:
    KinoSearch 0.165    <-- branches/maint_0.1x/
    KinoSearch 0.30_083 <-- trunk[1]

  CPAN after:
    KinoSearch1 1.00    <-- branches/ks1
    KinoSearch  0.31    <-- trunk

However, I think we should put out a new stable release soonish, so that
people who need API stability don't have to go all the way back to KinoSearch1
(no field-based sorting, much slower, etc.).  After that happens, CPAN might
look like this:

    KinoSearch1 1.00    <-- branches/ks1
    KinoSearch3 3.00    <-- branches/ks3
    KinoSearch  0.40    <-- trunk

> I don't have the same personal commitment to back compat for my own modules,
> so I'll happy continue pegging them to whatever the latest CPAN version of
> KinoSearch is. 

Right, this is how we inconvenience active developers like yourself the least.
You aren't required to back-fork.  

However, if somebody else wants your module bad enough, they'll probably fork
it for you.  :)

> Until, of course, I hear from users of my module who can't/won't
> upgrade their KS installs and then I'm fine with a back-fork under a different
> namespace, as you're describing (I think).

And presumably the maintenance burden of that back-forked module would be
minimal, because the library that it's pinned to won't be moving.

> > Does anybody know if any other projects have ever used a similar release
> > strategy?
> 
> I don't. Usually a project with abandons earlier user base by changing APIs at
> major releases levels, 

Right.  As discussed, I prefer not to leave formerly-active users in the
lurch.  However, your point suggests a slight variant on the plan above.

I'd been thinking that "KinoSearch" would have a perpetually unstable API, and
that we'd abandon all pretense that it would ever be stable.  However, we
could do as you say here and break API compatibility only at major release
points -- however, to avoid abandoning the earlier user base completely, we'd
spin off a renamed stable branch at each back-compat break.

If we did that, CPAN would look like this right away:

    KinoSearch1 1.00    <-- branches/ks1
    KinoSearch  1.00    <-- identical to branches/ks1 except for namespace
    KinoSearch  3.00_01 <-- trunk

We would then work to get trunk ready for a real version 3 release.  Once it
was published, CPAN would look like this:

    KinoSearch1 1.00    <-- branches/ks1
    KinoSearch  3.00    <-- branches/maint-3.x
    KinoSearch  4.00_01 <-- trunk

Then, in a year or so[2] when we release KinoSearch 4, CPAN would look like
this:

    KinoSearch1 1.00    <-- branches/ks1
    KinoSearch3 3.00    <-- branches/ks3
    KinoSearch  4.00    <-- branches/maint-4.x
    KinoSearch  5.00_01 <-- trunk

I don't like that as much, though, for a couple reasons.  

First, it would further put off the day when the current trunk gets published.
You wouldn't like that, nor would my colleagues here at Eventful, nor would I.
:)

Second, we'd have to hide development behind dev releases again -- I think it
would be too confusing to publish a distribution with a fluid API but a
version number greater than 1.0.

Hiding development behind dev releases thwarts one of the main rationales for
this plan: privileging active users over inactive ones.  We want as many
people as possible to be using svn trunk.

Generally, I suspect new users will find "KinoSearch" first.  If they see a
version number less than 1.0, they can't say they weren't warned when the API
changes out from underneath them.   Those for whom backwards compatibility is
important will then have the option of moving their code to a stable fork.
There may be features that aren't available on the stable fork, but that's
what privileging active users means!

Some people will discover "KinoSearch3" at the same time as "KinoSearch" and
perhaps assume that KinoSearch3 is the latest and greatest.  However, we can
put a prominent note at the top of the main documentation page to set them
straight.

> or does gymnastics to provide eternal backwards compat
> (the latter seems to be what Perl does).

The back-compatibility debates that rage on p5p were a big influence on this
proposed policy.  

Every time somebody suggests dropping a little-used misfeature that's holding
Perl back, the conservatives win the day and it stays.  But then chromatic[3]
points out that the people who are benefitting most from stringent back-compat
are the people who aren't actively contributing any more.

Under the proposed policy, we privilege and reward our most active users and
contributors.  However, we don't completely screw over people who were once
active but have since become inactive.  They are required to make a one-time
sweep through their code to use the spun-off stable branch.

This way, projects like MojoMojo and Socialtext -- which depend on KinoSearch
but may not control exactly when Perl modules get upgraded on a machine
running their software -- will be able to switch to a stable branch like
KinoSearch1 and avoid breakage.

Marvin Humphrey

[1] Technically 0.30_083 is based off of branches/0.30_08x, but it's pretty
    darn close to trunk.

[2] This is theoretical because I expect we'll have transitioned from 
    KinoSearch to Lucy by then, but it's useful to think about it this way
    because the plan under discussion will probably be used by Lucy in the
    future.

[3] For those who don't know, "chromatic" is a prominent Perl developer and
    author.


Re: Stable releases under new namespaces

Posted by Peter Karman <pe...@peknet.com>.
Marvin Humphrey wrote on 3/9/10 10:57 PM:

> 
> This isn't a perfect solution, but I think it's at least a satisfactory way to
> resolve the current situation.  The "stable" branch has been highly stable for
> a long time, but those users were still warned by that prominent ALPHA warning
> at the top of the docs -- expecting them to make a one-time switch is
> acceptable.

I've been delaying in my response to this because (a) it took me awhile to grok
the idea, and (b) I've been trying to play through scenarios based on my desire
to keep my KS extension modules happy.

I *think* I understand the idea. I.e., KinoSearch 0.1x becomes KinoSearch1,
KinoSearch 0.3x becomes KinoSearch3, etc. Is that right?

I don't have the same personal commitment to back compat for my own modules, so
I'll happy continue pegging them to whatever the latest CPAN version of
KinoSearch is. Until, of course, I hear from users of my module who can't/won't
upgrade their KS installs and then I'm fine with a back-fork under a different
namespace, as you're describing (I think).


> 
> Does anybody know if any other projects have ever used a similar release
> strategy?

I don't. Usually a project with abandons earlier user base by changing APIs at
major releases levels, or does gymnastics to provide eternal backwards compat
(the latter seems to be what Perl does).


-- 
Peter Karman  .  http://peknet.com/  .  peter@peknet.com