You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Yonik Seeley <yo...@lucidimagination.com> on 2009/05/28 00:23:13 UTC
upcoming Solr release on Lucene 2.9-dev
We're aiming for a Solr release in the next few weeks (as usual, we're
6 months behind when we wanted to make the release).
The catch is that Solr depends on Lucene 2.9, and there have been a
*lot* of changes. We're currently on r779312 (upgraded topday).
I'll add a note to Solr to warn users from using new Lucene APIs in
plugins, and warn them to not upgrade the version of Lucene in Solr
themselves. Solr itself doesn't require API back compat from Lucene
of course... we'll figure out some way to get stuff to work.
So concerns what does that leave?
- Index format stability... I believe since 2.4, the only change has
been commit metadata? Any guesses if the format for this remain
stable until 2.9 is released?
- stability in general (i.e. not crashing or producing corrupt indexes)
Solr is using read-only readers, IndexReader.reopen(),
IndexReader.incRef(), etc. All deletes are via IndexWriter
We're also using the new Collector classes, new FieldComparator
classes including custom comparators. We're passing
docsScoredInOrder=true, and we also depend on docs coming back in
sorted order in some places.
Solr is *not* using NRT features.
Any thoughts or concerns? Any specific changes we should wait for?
-Yonik
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: upcoming Solr release on Lucene 2.9-dev
Posted by Michael McCandless <lu...@mikemccandless.com>.
The IndexWriter diagnostics (LUCENE-1654: recording Lucene version,
Java/OS version, etc into each segment created) also bumped the index
file format.
And LUCENE-1623 (fixing back-compat issue w/ field names that have
non-ascii characters) also bumped the format of the fnm file.
Offhand I can't think of other issues that might change the format...
but we do have a lot of open 2.9 issues so I guess it's possible.
Does Solr use NIOFSDirectory/MMAPDirectory by default?
> We're passing docsScoredInOrder=true
Excellent! I'm wanting Lucene to do the same, in IndexSearcher when
we create the collector, for most queries.
And
On Wed, May 27, 2009 at 6:23 PM, Yonik Seeley
<yo...@lucidimagination.com> wrote:
> We're aiming for a Solr release in the next few weeks (as usual, we're
> 6 months behind when we wanted to make the release).
> The catch is that Solr depends on Lucene 2.9, and there have been a
> *lot* of changes. We're currently on r779312 (upgraded topday).
>
> I'll add a note to Solr to warn users from using new Lucene APIs in
> plugins, and warn them to not upgrade the version of Lucene in Solr
> themselves. Solr itself doesn't require API back compat from Lucene
> of course... we'll figure out some way to get stuff to work.
>
> So concerns what does that leave?
> - Index format stability... I believe since 2.4, the only change has
> been commit metadata? Any guesses if the format for this remain
> stable until 2.9 is released?
> - stability in general (i.e. not crashing or producing corrupt indexes)
>
> Solr is using read-only readers, IndexReader.reopen(),
> IndexReader.incRef(), etc. All deletes are via IndexWriter
> We're also using the new Collector classes, new FieldComparator
> classes including custom comparators. We're passing
> docsScoredInOrder=true, and we also depend on docs coming back in
> sorted order in some places.
>
> Solr is *not* using NRT features.
>
> Any thoughts or concerns? Any specific changes we should wait for?
>
> -Yonik
> http://www.lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: upcoming Solr release on Lucene 2.9-dev
Posted by Grant Ingersoll <gs...@apache.org>.
On May 28, 2009, at 10:24 AM, Yonik Seeley wrote:
>
>
> removing all the deprecated stuff related to Token/TokenStream is for
> another day...
I think Lucene needs to re-look at the TokenStream stuff anyway. I
seem to recall talking with Michael B. about changing it at
ApacheCon. I'd be interested in feedback from people on whether they
have updated their custom work on it. I thought I was a post from
Shai about it. I'll try to write up some more formal thoughts on it
soon.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: upcoming Solr release on Lucene 2.9-dev
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Thu, May 28, 2009 at 2:56 AM, Shai Erera <se...@gmail.com> wrote:
> If by changes you also mean deprecated features, then take a look at
> LUCENE-1614 - if you have your own Scorers/DISIs, you might want to
> implement the new methods, since the current ones are deprecated.
Yes, we have our own Scorers, but changes to those, as well as
removing all the deprecated stuff related to Token/TokenStream is for
another day...
-Yonik
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: upcoming Solr release on Lucene 2.9-dev
Posted by Shai Erera <se...@gmail.com>.
If by changes you also mean deprecated features, then take a look at
LUCENE-1614 - if you have your own Scorers/DISIs, you might want to
implement the new methods, since the current ones are deprecated.
On Thu, May 28, 2009 at 1:23 AM, Yonik Seeley <yo...@lucidimagination.com>wrote:
> We're aiming for a Solr release in the next few weeks (as usual, we're
> 6 months behind when we wanted to make the release).
> The catch is that Solr depends on Lucene 2.9, and there have been a
> *lot* of changes. We're currently on r779312 (upgraded topday).
>
> I'll add a note to Solr to warn users from using new Lucene APIs in
> plugins, and warn them to not upgrade the version of Lucene in Solr
> themselves. Solr itself doesn't require API back compat from Lucene
> of course... we'll figure out some way to get stuff to work.
>
> So concerns what does that leave?
> - Index format stability... I believe since 2.4, the only change has
> been commit metadata? Any guesses if the format for this remain
> stable until 2.9 is released?
> - stability in general (i.e. not crashing or producing corrupt indexes)
>
> Solr is using read-only readers, IndexReader.reopen(),
> IndexReader.incRef(), etc. All deletes are via IndexWriter
> We're also using the new Collector classes, new FieldComparator
> classes including custom comparators. We're passing
> docsScoredInOrder=true, and we also depend on docs coming back in
> sorted order in some places.
>
> Solr is *not* using NRT features.
>
> Any thoughts or concerns? Any specific changes we should wait for?
>
> -Yonik
> http://www.lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
Re: upcoming Solr release on Lucene 2.9-dev
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Thu, May 28, 2009 at 5:35 AM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> The IndexWriter diagnostics (LUCENE-1654: recording Lucene version,
> Java/OS version, etc into each segment created) also bumped the index
> file format.
>
> And LUCENE-1623 (fixing back-compat issue w/ field names that have
> non-ascii characters) also bumped the format of the fnm file.
>
> Offhand I can't think of other issues that might change the format...
> but we do have a lot of open 2.9 issues so I guess it's possible.
>
> Does Solr use NIOFSDirectory/MMAPDirectory by default?
Yep, Solr defaults to FSDir on Windows and NIO on anything else.
-Yonik
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org