lucene releases vs trunk

[Yonik Seeley <yo...@lucidimagination.com>]

For the next release cycle (presumably 1.5?) I think we should really
try to stick to released versions of Lucene, and not use dev/trunk
versions.
Early in Solr's lifetime, Lucene trunk was more stable (APIs changed
little, even on non-released versions), and Lucene releases were few
and far between.
Today, the pace of change in Lucene has quickened, and Lucene APIs are
much more in flux until a release is made.  It's also now harder to
support a Lucene dev release given the growth in complexity
(particularly for indexing code).  Releases are made more often too,
making using released versions more practical.
Many of our users dislike our use of dev versions of Lucene too.

And yes, 1.4 isn't out the door yet - but people often tend to hit the
ground running on the next release.

-Yonik
http://www.lucidimagination.com

Re: lucene releases vs trunk

[Yonik Seeley <yo...@lucidimagination.com>]

On Thu, Jun 25, 2009 at 1:29 PM, Chris
Hostetter<ho...@fucit.org> wrote:
> : This proposal was just for the next (1.5?) release cycle though.
>        ...
> : I agree though - there is rapid movement in Lucene these days, and things can
> : be pulled back or altered fairly easily during trunk dev. Sometimes even index
> : format changing issues - which can be a real pain (having suffered that first
> : hand in the past). The closer we can stay to actual Lucene releases in
> : general, the better I think.
>
> I suggest we not worry about it too much until the situation arrises.

I'm calling attention to it because I don't believe the move to
2.9-dev was ever discussed on solr-dev.
AFAIK it was committed as part of SOLR-805... something I missed, and
I doubt I'm the only one.

The default should be to use released Lucene versions, and we should
reluctantly move off of that.

> Once upon a time the decision to bump the lucene-java rev in Solr was
> drien largely based on wether we people that that version was had useful
> additions *and* was relatively "solid".  My impression more recently is
> that people have been bumping the rev primarily with the
> features/improvements in mind, and less consideration of the stability --
> probably due to the (completely valid) assumption that solr trunk doesn't
> *need* to be any more stable then the lucene-java trunk, so we might as
> well go ahead and rev and help shake things out.

Right - if we're relatively sure that a Lucene release is imminent
(and will happen before a Solr release), it's not such a bad idea to
upgrade.

-Yonik
http://www.lucidimagination.com

Re: lucene releases vs trunk

[Chris Hostetter <ho...@fucit.org>]

: This proposal was just for the next (1.5?) release cycle though.
	...
: I agree though - there is rapid movement in Lucene these days, and things can
: be pulled back or altered fairly easily during trunk dev. Sometimes even index
: format changing issues - which can be a real pain (having suffered that first
: hand in the past). The closer we can stay to actual Lucene releases in
: general, the better I think.

I suggest we not worry about it too much until the situation arrises.

Once upon a time the decision to bump the lucene-java rev in Solr was 
drien largely based on wether we people that that version was had useful 
additions *and* was relatively "solid".  My impression more recently is 
that people have been bumping the rev primarily with the 
features/improvements in mind, and less consideration of the stability -- 
probably due to the (completely valid) assumption that solr trunk doesn't 
*need* to be any more stable then the lucene-java trunk, so we might as 
well go ahead and rev and help shake things out.

If we go back in hindsight with and imagine that Solr might have be 
released at any given moment, then some of those lucene-java revs would 
probably seem less prudent then others.

Moving forward we should just proceed the same way: don't rev any Solr 
deps (lucene-java or otherwise) unless you're confident enough in the new 
version that you'd be ready to vote for a release that includes it.

If lucene-java maintains a really heavy state of churn on the trunk, then 
nightlies probably aren't going to meet that criteria for us moving 
forward -- but if, at some point in future, someone wants to bump the rev 
to a nightly that is exactly the same as lucene-java 3.2.3 plus one new 
TokenFilter they want to write a Factory for ... i really don't see that 
as being less stable then the "stable" 3.2.3 version.


Common sense makes the most sense.


-Hoss

Re: lucene releases vs trunk

[Mark Miller <ma...@gmail.com>]

This proposal was just for the next (1.5?) release cycle though.

Lucene 2.9 is going to beat Solr 1.4 in any case I think, so it would 
seem to make sense to go with it, and so this won't be an issue for this 
release. For 1.5, we can just run as normal, and I guess, and plan to 
release a bit after the next version of Lucene releases. Then take it a 
release at a time based on how things go in Lucene land?

I agree though - there is rapid movement in Lucene these days, and 
things can be pulled back or altered fairly easily during trunk dev. 
Sometimes even index format changing issues - which can be a real pain 
(having suffered that first hand in the past). The closer we can stay to 
actual Lucene releases in general, the better I think.

-- 
- Mark

http://www.lucidimagination.com


Otis Gospodnetic wrote:
> I kind of agree... But will this (not) affect how quickly new features in Luceneland will get their Solr support?  In other words, if we have to wait for a proper Lucene release, doesn't that mean that:
>
> 1) Solr releases will depend on Lucene releases (unless there are some Solr-only changes that don't depend on newer version of Lucene)
> 2) Solr releases will lag Lucene releases quite a bit because only after Lucene has been released Solr developers/contributors will be able to start work on integrating new Lucene features into Solr?
>
>
> Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>
>
>
> ----- Original Message ----
>   
>> From: Yonik Seeley <yo...@lucidimagination.com>
>> To: solr-dev@lucene.apache.org
>> Sent: Thursday, June 25, 2009 7:18:31 AM
>> Subject: lucene releases vs trunk
>>
>> For the next release cycle (presumably 1.5?) I think we should really
>> try to stick to released versions of Lucene, and not use dev/trunk
>> versions.
>> Early in Solr's lifetime, Lucene trunk was more stable (APIs changed
>> little, even on non-released versions), and Lucene releases were few
>> and far between.
>> Today, the pace of change in Lucene has quickened, and Lucene APIs are
>> much more in flux until a release is made.  It's also now harder to
>> support a Lucene dev release given the growth in complexity
>> (particularly for indexing code).  Releases are made more often too,
>> making using released versions more practical.
>> Many of our users dislike our use of dev versions of Lucene too.
>>
>> And yes, 1.4 isn't out the door yet - but people often tend to hit the
>> ground running on the next release.
>>
>> -Yonik
>> http://www.lucidimagination.com
>>     
>
>   

Re: lucene releases vs trunk

[Otis Gospodnetic <ot...@yahoo.com>]

I kind of agree... But will this (not) affect how quickly new features in Luceneland will get their Solr support?  In other words, if we have to wait for a proper Lucene release, doesn't that mean that:

1) Solr releases will depend on Lucene releases (unless there are some Solr-only changes that don't depend on newer version of Lucene)
2) Solr releases will lag Lucene releases quite a bit because only after Lucene has been released Solr developers/contributors will be able to start work on integrating new Lucene features into Solr?


Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: Yonik Seeley <yo...@lucidimagination.com>
> To: solr-dev@lucene.apache.org
> Sent: Thursday, June 25, 2009 7:18:31 AM
> Subject: lucene releases vs trunk
> 
> For the next release cycle (presumably 1.5?) I think we should really
> try to stick to released versions of Lucene, and not use dev/trunk
> versions.
> Early in Solr's lifetime, Lucene trunk was more stable (APIs changed
> little, even on non-released versions), and Lucene releases were few
> and far between.
> Today, the pace of change in Lucene has quickened, and Lucene APIs are
> much more in flux until a release is made.  It's also now harder to
> support a Lucene dev release given the growth in complexity
> (particularly for indexing code).  Releases are made more often too,
> making using released versions more practical.
> Many of our users dislike our use of dev versions of Lucene too.
> 
> And yes, 1.4 isn't out the door yet - but people often tend to hit the
> ground running on the next release.
> 
> -Yonik
> http://www.lucidimagination.com

Re: lucene releases vs trunk

[Walter Underwood <wu...@netflix.com>]

This is an excellent idea.

When I find a problem and want to research the Lucene bugs that might
describe it, that is really hard with a trunk build. It's easy with a
release build.

wunder

On 6/25/09 4:18 AM, "Yonik Seeley" <yo...@lucidimagination.com> wrote:

> For the next release cycle (presumably 1.5?) I think we should really
> try to stick to released versions of Lucene, and not use dev/trunk
> versions.
> Early in Solr's lifetime, Lucene trunk was more stable (APIs changed
> little, even on non-released versions), and Lucene releases were few
> and far between.
> Today, the pace of change in Lucene has quickened, and Lucene APIs are
> much more in flux until a release is made.  It's also now harder to
> support a Lucene dev release given the growth in complexity
> (particularly for indexing code).  Releases are made more often too,
> making using released versions more practical.
> Many of our users dislike our use of dev versions of Lucene too.
> 
> And yes, 1.4 isn't out the door yet - but people often tend to hit the
> ground running on the next release.
> 
> -Yonik
> http://www.lucidimagination.com