You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Mark Miller <ma...@gmail.com> on 2009/08/28 21:02:26 UTC
Lucene 2.9 RC2 now available for testing
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello Lucene users,
On behalf of the Lucene dev community (a growing community far larger
than just the committers) I would like to announce the second release
candidate for Lucene 2.9.
Please download and check it out – take it for a spin and kick the
tires. If all goes well, we hope to release the final version of
Lucene 2.9 in a little over a week.
The following changes have been applied since the first release candidate:
LUCENE-1867: replace collation/lib/icu4j.jar with a smaller icu jar
LUCENE-1868: add Arabic stemmer to notice.txt
LUCENE-1870: fix contrib dist - missing analyzers/db binaries - extra
miscellaneous folder with misc readme in it
LUCENE-1869: include 'file exists?' when we throw RuntimeException
from fdx or tvx size mismatches during flush or merge
LUCENE-1871: Add option to avoid needlessly wrapping TokenStream with
CachingTokenFilter in Highlighter
While we generally try and maintain full backwards compatibility
between major versions, Lucene 2.9 has a variety of breaks that are
spelled out in the 'Changes in backwards compatibility policy' section
of CHANGES.txt.
We recommend that you recompile your application with Lucene 2.9
rather than attempting to “drop” it in. This will alert you to any
issues you may have to fix if you are affected by one of the backward
compatibility breaks. As always, its a really good idea to thoroughly
read CHANGES.txt before upgrading. Also, remember that this is a
release candidate, and not the final Lucene 2.9 release.
Lucene 2.9 comes with a bevy of new features, including:
Per segment searching and caching (can lead to much faster reopen
among other things)
Near real-time search capabilities added to IndexWriter
New queries, including NumericRangeQuery and NumericRangeFilter –
fast, highly scalable alternatives to RangeQuery/RangeFilter for
numeric searches.
Smarter, more scalable multi-term queries (wildcard, range, etc)
A freshly optimized Collector/Scorer API
Improved Unicode support
A new Attribute based TokenStream API
A new QueryParser framework in contrib with a core QueryParser
replacement impl included.
….
And many, many more features, bug fixes, optimizations, and various
improvements. You can find the full list of changes here:
HTML version:
http://people.apache.org/~markrmiller/staging-area/lucene2.9changes/Changes.html
Text version:
http://people.apache.org/~markrmiller/staging-area/lucene2.9changes/CHANGES.txt
Many changes have also occurred in Lucene's contrib area:
http://people.apache.org/~markrmiller/staging-area/lucene2.9changes/contrib/CHANGES.txt
Download release candidate 1 here:
http://people.apache.org/~markrmiller/staging-area/lucene2.9rc2/
Be sure to report back with any issues you find!
Thanks,
Mark Miller
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkqYKcIACgkQ0DU3IV7ywDkZHACfbzHKac0sVjNkdSi+79WTYWme
JR4An18v1SJ6HN8mkCYHF0ybqUSypOG/
=qS9t
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene 2.9 RC2 now available for testing
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Wed, Sep 9, 2009 at 9:17 AM, Yonik
Seeley<yo...@lucidimagination.com> wrote:
> On Wed, Sep 9, 2009 at 8:57 AM, Peter Keegan<pe...@gmail.com> wrote:
>> Using JProfiler, I observe that the improvement
>> is due to a huge reduction in the number of calls to TermDocs.next and
>> TermDocs.skipTo (about 65% fewer calls).
>
> Indexes are searched per-segment now (i.e. MultiTermDocs isn't normally used).
> Off the top of my head, I'm not sure how this can lead to fewer
> TermDocs.skipTo() calls though.
Wait... perhaps it's just that accounting for the skipTo() decrease?
Instead of MultiTermDocs.skipTo() delegating to
SegmentTermDocs.skipTo() (2 calls since they both inherit from
TermDocs), it's now just SegmentTermDocs.skipTo() directly.
-Yonik
http://www.lucidimagination.com
Are you sure you weren't also
> counting Scorer.skipTo()... which would now be Scorer.advance()?
> Have you verified that your custom scorer is working correctly with
> 2.9 and that you're getting the same number of hits on the overall
> query as you were with previous versions?
>
> -Yonik
> http://www.lucidimagination.com
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene 2.9 RC2 now available for testing
Posted by Peter Keegan <pe...@gmail.com>.
> http://svn.apache.org/viewvc?view=rev&revision=630698
This may be it. The scorer is sparse and usually in a conjuction with a
dense scorer.
Does the index format matter? I haven't yet built it with 2.9.
Peter
On Wed, Sep 9, 2009 at 10:17 AM, Yonik Seeley <yo...@lucidimagination.com>wrote:
> On Wed, Sep 9, 2009 at 9:40 AM, Peter Keegan<pe...@gmail.com>
> wrote:
> > IndexSearcher.search is calling my custom scorer's 'next' and 'doc'
> methods
> > 64% fewer times. I see no 'advance' method in any of the hot spots'. I am
> > getting the same number of hits from the custom scorer.
> > Has the BooleanScorer2 logic changed?
>
> What does your query structure look like?
> BS2 hasn't changed much. The old BS may be used in certain
> circumstances, but not in a way that you would see a decrease in
> skipTo()+next().
>
> Shot in the dark: if your scorer is part of a conjunction, maybe
> you're getting lucky with an optimization:
> http://svn.apache.org/viewvc?view=rev&revision=630698
> It was part of http://issues.apache.org/jira/browse/LUCENE-693 , but I
> had a slight bug to the "highest skip first" optimization that wasn't
> fixed until 2.4
>
> Let's assume that your custom scorer is dense (matches almost
> everything) and that another scorer is sparse. If your custom scorer
> appears first, the conjunction scorer logic would be like so:
> custom.skipTo(), scorer1.skipTo(), custom.skipTo() => match!
>
> If the sparse scorer is first, it will look more like so:
> scorer1.skipTo(), custom.skipTo() => match!
>
> -Yonik
> http://www.lucidimagination.com
>
>
> > Peter
> >
> > On Wed, Sep 9, 2009 at 9:17 AM, Yonik Seeley <
> > yonik.seeley@lucidimagination.com> wrote:
> >
> >> On Wed, Sep 9, 2009 at 8:57 AM, Peter Keegan<pe...@gmail.com>
> >> wrote:
> >> > Using JProfiler, I observe that the improvement
> >> > is due to a huge reduction in the number of calls to TermDocs.next and
> >> > TermDocs.skipTo (about 65% fewer calls).
> >>
> >> Indexes are searched per-segment now (i.e. MultiTermDocs isn't normally
> >> used).
> >> Off the top of my head, I'm not sure how this can lead to fewer
> >> TermDocs.skipTo() calls though. Are you sure you weren't also
> >> counting Scorer.skipTo()... which would now be Scorer.advance()?
> >> Have you verified that your custom scorer is working correctly with
> >> 2.9 and that you're getting the same number of hits on the overall
> >> query as you were with previous versions?
> >>
> >> -Yonik
> >> http://www.lucidimagination.com
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Lucene 2.9 RC2 now available for testing
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Wed, Sep 9, 2009 at 9:40 AM, Peter Keegan<pe...@gmail.com> wrote:
> IndexSearcher.search is calling my custom scorer's 'next' and 'doc' methods
> 64% fewer times. I see no 'advance' method in any of the hot spots'. I am
> getting the same number of hits from the custom scorer.
> Has the BooleanScorer2 logic changed?
What does your query structure look like?
BS2 hasn't changed much. The old BS may be used in certain
circumstances, but not in a way that you would see a decrease in
skipTo()+next().
Shot in the dark: if your scorer is part of a conjunction, maybe
you're getting lucky with an optimization:
http://svn.apache.org/viewvc?view=rev&revision=630698
It was part of http://issues.apache.org/jira/browse/LUCENE-693 , but I
had a slight bug to the "highest skip first" optimization that wasn't
fixed until 2.4
Let's assume that your custom scorer is dense (matches almost
everything) and that another scorer is sparse. If your custom scorer
appears first, the conjunction scorer logic would be like so:
custom.skipTo(), scorer1.skipTo(), custom.skipTo() => match!
If the sparse scorer is first, it will look more like so:
scorer1.skipTo(), custom.skipTo() => match!
-Yonik
http://www.lucidimagination.com
> Peter
>
> On Wed, Sep 9, 2009 at 9:17 AM, Yonik Seeley <
> yonik.seeley@lucidimagination.com> wrote:
>
>> On Wed, Sep 9, 2009 at 8:57 AM, Peter Keegan<pe...@gmail.com>
>> wrote:
>> > Using JProfiler, I observe that the improvement
>> > is due to a huge reduction in the number of calls to TermDocs.next and
>> > TermDocs.skipTo (about 65% fewer calls).
>>
>> Indexes are searched per-segment now (i.e. MultiTermDocs isn't normally
>> used).
>> Off the top of my head, I'm not sure how this can lead to fewer
>> TermDocs.skipTo() calls though. Are you sure you weren't also
>> counting Scorer.skipTo()... which would now be Scorer.advance()?
>> Have you verified that your custom scorer is working correctly with
>> 2.9 and that you're getting the same number of hits on the overall
>> query as you were with previous versions?
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene 2.9 RC2 now available for testing
Posted by Peter Keegan <pe...@gmail.com>.
>Is it possible that skipTo is very costly with your custom scorer?
It's no more expensive than 'next'. The scorer's 'skipTo' and 'next' methods
call termdocs.skipTo or termdocs.next to get the next 'candidate' doc. This
just checks a BitVector to find the next non-deleted doc. But the scorer
must visit some metadata (RAM resident) for each candidate doc, which is the
most expensive part. So, reducing the number of calls to the scorer's 'next'
and 'skipTo' methods is always a win. Since the custom scorer is the
bottleneck with both versions of Lucene (according to JProfiler), the
improvements here are going weigh much more.
Peter
On Wed, Sep 9, 2009 at 9:57 AM, Michael McCandless <
lucene@mikemccandless.com> wrote:
> Right, BooleanQuery will now try to use BooleanScorer (does "out of
> order" collection, which does not use skipTo/advance at all, I think)
> when possible, instead of BooleanScorer2.
>
> This only applies for boolean queries that have only SHOULD clauses,
> and up to 32 MUST_NOT clauses (if there's even 1 MUST clause,
> BooleanScorer2 will be used).
>
> But: it's interesting that you see such gains (2x-10x) from this. In
> simple (clauses that were TermQuery) boolean queries I saw maybe ~30%
> speedup, I think.
>
> Is it possible that skipTo is very costly with your custom scorer?
> That could explain the gains.
>
> Mike
>
> On Wed, Sep 9, 2009 at 9:44 AM, Mark Miller<ma...@gmail.com> wrote:
> > How about the new score inorder/out of order stuff? It was an option
> > before, but I think now it uses whats best by default? And pairs with
> > the collector? I didn't follow any of that closely though.
> >
> > - Mark
> >
> > Peter Keegan wrote:
> >> IndexSearcher.search is calling my custom scorer's 'next' and 'doc'
> methods
> >> 64% fewer times. I see no 'advance' method in any of the hot spots'. I
> am
> >> getting the same number of hits from the custom scorer.
> >> Has the BooleanScorer2 logic changed?
> >>
> >> Peter
> >>
> >> On Wed, Sep 9, 2009 at 9:17 AM, Yonik Seeley <
> >> yonik.seeley@lucidimagination.com> wrote:
> >>
> >>
> >>> On Wed, Sep 9, 2009 at 8:57 AM, Peter Keegan<pe...@gmail.com>
> >>> wrote:
> >>>
> >>>> Using JProfiler, I observe that the improvement
> >>>> is due to a huge reduction in the number of calls to TermDocs.next and
> >>>> TermDocs.skipTo (about 65% fewer calls).
> >>>>
> >>> Indexes are searched per-segment now (i.e. MultiTermDocs isn't normally
> >>> used).
> >>> Off the top of my head, I'm not sure how this can lead to fewer
> >>> TermDocs.skipTo() calls though. Are you sure you weren't also
> >>> counting Scorer.skipTo()... which would now be Scorer.advance()?
> >>> Have you verified that your custom scorer is working correctly with
> >>> 2.9 and that you're getting the same number of hits on the overall
> >>> query as you were with previous versions?
> >>>
> >>> -Yonik
> >>> http://www.lucidimagination.com
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>>
> >>>
> >>>
> >>
> >>
> >
> >
> > --
> > - Mark
> >
> > http://www.lucidimagination.com
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Lucene 2.9 RC2 now available for testing
Posted by Michael McCandless <lu...@mikemccandless.com>.
Right, BooleanQuery will now try to use BooleanScorer (does "out of
order" collection, which does not use skipTo/advance at all, I think)
when possible, instead of BooleanScorer2.
This only applies for boolean queries that have only SHOULD clauses,
and up to 32 MUST_NOT clauses (if there's even 1 MUST clause,
BooleanScorer2 will be used).
But: it's interesting that you see such gains (2x-10x) from this. In
simple (clauses that were TermQuery) boolean queries I saw maybe ~30%
speedup, I think.
Is it possible that skipTo is very costly with your custom scorer?
That could explain the gains.
Mike
On Wed, Sep 9, 2009 at 9:44 AM, Mark Miller<ma...@gmail.com> wrote:
> How about the new score inorder/out of order stuff? It was an option
> before, but I think now it uses whats best by default? And pairs with
> the collector? I didn't follow any of that closely though.
>
> - Mark
>
> Peter Keegan wrote:
>> IndexSearcher.search is calling my custom scorer's 'next' and 'doc' methods
>> 64% fewer times. I see no 'advance' method in any of the hot spots'. I am
>> getting the same number of hits from the custom scorer.
>> Has the BooleanScorer2 logic changed?
>>
>> Peter
>>
>> On Wed, Sep 9, 2009 at 9:17 AM, Yonik Seeley <
>> yonik.seeley@lucidimagination.com> wrote:
>>
>>
>>> On Wed, Sep 9, 2009 at 8:57 AM, Peter Keegan<pe...@gmail.com>
>>> wrote:
>>>
>>>> Using JProfiler, I observe that the improvement
>>>> is due to a huge reduction in the number of calls to TermDocs.next and
>>>> TermDocs.skipTo (about 65% fewer calls).
>>>>
>>> Indexes are searched per-segment now (i.e. MultiTermDocs isn't normally
>>> used).
>>> Off the top of my head, I'm not sure how this can lead to fewer
>>> TermDocs.skipTo() calls though. Are you sure you weren't also
>>> counting Scorer.skipTo()... which would now be Scorer.advance()?
>>> Have you verified that your custom scorer is working correctly with
>>> 2.9 and that you're getting the same number of hits on the overall
>>> query as you were with previous versions?
>>>
>>> -Yonik
>>> http://www.lucidimagination.com
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>>
>>
>
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene 2.9 RC2 now available for testing
Posted by Mark Miller <ma...@gmail.com>.
How about the new score inorder/out of order stuff? It was an option
before, but I think now it uses whats best by default? And pairs with
the collector? I didn't follow any of that closely though.
- Mark
Peter Keegan wrote:
> IndexSearcher.search is calling my custom scorer's 'next' and 'doc' methods
> 64% fewer times. I see no 'advance' method in any of the hot spots'. I am
> getting the same number of hits from the custom scorer.
> Has the BooleanScorer2 logic changed?
>
> Peter
>
> On Wed, Sep 9, 2009 at 9:17 AM, Yonik Seeley <
> yonik.seeley@lucidimagination.com> wrote:
>
>
>> On Wed, Sep 9, 2009 at 8:57 AM, Peter Keegan<pe...@gmail.com>
>> wrote:
>>
>>> Using JProfiler, I observe that the improvement
>>> is due to a huge reduction in the number of calls to TermDocs.next and
>>> TermDocs.skipTo (about 65% fewer calls).
>>>
>> Indexes are searched per-segment now (i.e. MultiTermDocs isn't normally
>> used).
>> Off the top of my head, I'm not sure how this can lead to fewer
>> TermDocs.skipTo() calls though. Are you sure you weren't also
>> counting Scorer.skipTo()... which would now be Scorer.advance()?
>> Have you verified that your custom scorer is working correctly with
>> 2.9 and that you're getting the same number of hits on the overall
>> query as you were with previous versions?
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>
>
>
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene 2.9 RC2 now available for testing
Posted by Peter Keegan <pe...@gmail.com>.
IndexSearcher.search is calling my custom scorer's 'next' and 'doc' methods
64% fewer times. I see no 'advance' method in any of the hot spots'. I am
getting the same number of hits from the custom scorer.
Has the BooleanScorer2 logic changed?
Peter
On Wed, Sep 9, 2009 at 9:17 AM, Yonik Seeley <
yonik.seeley@lucidimagination.com> wrote:
> On Wed, Sep 9, 2009 at 8:57 AM, Peter Keegan<pe...@gmail.com>
> wrote:
> > Using JProfiler, I observe that the improvement
> > is due to a huge reduction in the number of calls to TermDocs.next and
> > TermDocs.skipTo (about 65% fewer calls).
>
> Indexes are searched per-segment now (i.e. MultiTermDocs isn't normally
> used).
> Off the top of my head, I'm not sure how this can lead to fewer
> TermDocs.skipTo() calls though. Are you sure you weren't also
> counting Scorer.skipTo()... which would now be Scorer.advance()?
> Have you verified that your custom scorer is working correctly with
> 2.9 and that you're getting the same number of hits on the overall
> query as you were with previous versions?
>
> -Yonik
> http://www.lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Lucene 2.9 RC2 now available for testing
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Wed, Sep 9, 2009 at 8:57 AM, Peter Keegan<pe...@gmail.com> wrote:
> Using JProfiler, I observe that the improvement
> is due to a huge reduction in the number of calls to TermDocs.next and
> TermDocs.skipTo (about 65% fewer calls).
Indexes are searched per-segment now (i.e. MultiTermDocs isn't normally used).
Off the top of my head, I'm not sure how this can lead to fewer
TermDocs.skipTo() calls though. Are you sure you weren't also
counting Scorer.skipTo()... which would now be Scorer.advance()?
Have you verified that your custom scorer is working correctly with
2.9 and that you're getting the same number of hits on the overall
query as you were with previous versions?
-Yonik
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene 2.9 RC2 now available for testing
Posted by Peter Keegan <pe...@gmail.com>.
I've been testing 2.9 RC2 lately and comparing query performance to 2.3.2.
I'm seeing a huge increase in throughput (2x-10x) on an index that was built
with 2.3.2. The queries have a lot of BoostingTermQuerys and boolean clauses
containing a custom scorer. Using JProfiler, I observe that the improvement
is due to a huge reduction in the number of calls to TermDocs.next and
TermDocs.skipTo (about 65% fewer calls). Since the custom scorer is somewhat
expensive and frequent in these queries, the improvement is quite noticable.
Have there been a lot of optimizations in the IndexSearcher/skipto/next
logic since 2.3.2?
Should I expect to see even more query improvement by building the index
with 2.9? (pinch me!)
Good stuff!
Thanks,
Peter
On Fri, Aug 28, 2009 at 3:02 PM, Mark Miller <ma...@gmail.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hello Lucene users,
>
> On behalf of the Lucene dev community (a growing community far larger
> than just the committers) I would like to announce the second release
> candidate for Lucene 2.9.
>
> Please download and check it out – take it for a spin and kick the
> tires. If all goes well, we hope to release the final version of
> Lucene 2.9 in a little over a week.
>
> The following changes have been applied since the first release candidate:
> LUCENE-1867: replace collation/lib/icu4j.jar with a smaller icu jar
> LUCENE-1868: add Arabic stemmer to notice.txt
> LUCENE-1870: fix contrib dist - missing analyzers/db binaries - extra
> miscellaneous folder with misc readme in it
> LUCENE-1869: include 'file exists?' when we throw RuntimeException
> from fdx or tvx size mismatches during flush or merge
> LUCENE-1871: Add option to avoid needlessly wrapping TokenStream with
> CachingTokenFilter in Highlighter
>
> While we generally try and maintain full backwards compatibility
> between major versions, Lucene 2.9 has a variety of breaks that are
> spelled out in the 'Changes in backwards compatibility policy' section
> of CHANGES.txt.
>
> We recommend that you recompile your application with Lucene 2.9
> rather than attempting to “drop” it in. This will alert you to any
> issues you may have to fix if you are affected by one of the backward
> compatibility breaks. As always, its a really good idea to thoroughly
> read CHANGES.txt before upgrading. Also, remember that this is a
> release candidate, and not the final Lucene 2.9 release.
>
> Lucene 2.9 comes with a bevy of new features, including:
>
> Per segment searching and caching (can lead to much faster reopen
> among other things)
>
> Near real-time search capabilities added to IndexWriter
>
> New queries, including NumericRangeQuery and NumericRangeFilter –
> fast, highly scalable alternatives to RangeQuery/RangeFilter for
> numeric searches.
>
> Smarter, more scalable multi-term queries (wildcard, range, etc)
>
> A freshly optimized Collector/Scorer API
>
> Improved Unicode support
>
> A new Attribute based TokenStream API
>
> A new QueryParser framework in contrib with a core QueryParser
> replacement impl included.
>
> ….
>
> And many, many more features, bug fixes, optimizations, and various
> improvements. You can find the full list of changes here:
>
> HTML version:
>
> http://people.apache.org/~markrmiller/staging-area/lucene2.9changes/Changes.html<http://people.apache.org/%7Emarkrmiller/staging-area/lucene2.9changes/Changes.html>
>
> Text version:
>
> http://people.apache.org/~markrmiller/staging-area/lucene2.9changes/CHANGES.txt<http://people.apache.org/%7Emarkrmiller/staging-area/lucene2.9changes/CHANGES.txt>
>
> Many changes have also occurred in Lucene's contrib area:
>
> http://people.apache.org/~markrmiller/staging-area/lucene2.9changes/contrib/CHANGES.txt<http://people.apache.org/%7Emarkrmiller/staging-area/lucene2.9changes/contrib/CHANGES.txt>
>
> Download release candidate 1 here:
> http://people.apache.org/~markrmiller/staging-area/lucene2.9rc2/<http://people.apache.org/%7Emarkrmiller/staging-area/lucene2.9rc2/>
>
> Be sure to report back with any issues you find!
>
> Thanks,
>
> Mark Miller
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkqYKcIACgkQ0DU3IV7ywDkZHACfbzHKac0sVjNkdSi+79WTYWme
> JR4An18v1SJ6HN8mkCYHF0ybqUSypOG/
> =qS9t
> -----END PGP SIGNATURE-----
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Lucene 2.9 RC2 now available for testing
Posted by Marcelo Ochoa <ma...@gmail.com>.
Hi All:
I am already have integrated Lucene 2.9RC2 with Lucene Domain Index:
http://docs.google.com/Doc?id=ddgw7sjp_54fgj9kg
As usual, a new Lucene version do a fastest product :)
All my internal test runs OK and I only need to re-test on 10g database.
Once Lucene 2.9 is ready for production I'll put a new binary
distribution of Lucene Domain Index, updating the docs to reflect the
new functionality of TopFieldCollector which is available from the SQL
layer through the combination of lscore() and lcontains(,,'sort',..)
operators.
Thanks a lot to all the dev group to release a new great piece of software.
Best regards, Marcelo.
On Fri, Aug 28, 2009 at 4:18 PM, Mark Miller<ma...@gmail.com> wrote:
> Mark Miller wrote:
>>
>> Download release candidate 1 here:
>> http://people.apache.org/~markrmiller/staging-area/lucene2.9rc2/
>>
> In case anyone catches - yes that is a cut and paste typo - should read
> release candidate 2 (obvious, but just to cross my t's).
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
Marcelo F. Ochoa
http://marceloochoa.blogspot.com/
http://marcelo.ochoa.googlepages.com/home
______________
Want to integrate Lucene and Oracle?
http://marceloochoa.blogspot.com/2007/09/running-lucene-inside-your-oracle-jvm.html
Is Oracle 11g REST ready?
http://marceloochoa.blogspot.com/2008/02/is-oracle-11g-rest-ready.html
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Porting Java Lucene 2.9 to Lucene.Net (was: RE: Lucene 2.9 RC2 now available for testing)
Posted by George Aroush <ge...@aroush.net>.
> -----Original Message-----
> From: Chris Hostetter [mailto:hossman_lucene@fucit.org]
> Sent: Monday, August 31, 2009 1:00 AM
> To: java-dev@lucene.apache.org
> Subject: Re: Porting Java Lucene 2.9 to Lucene.Net (was: RE: Lucene 2.9
RC2 now available for testing)
>
>
> : My question is, I would prefer to track SVN commits to keep track of
> : changes, vs. what I'm doing now. This will allow us to stay weeks
> : behind a Java release vs. months or years as it is now. However, while
> : I'm subscribed to SVN's commits mailing list, I'm not getting all those
> : commits! For example, a commit made this past Friday, I never got an
> : email for, while other commits I do. Any idea what maybe going on?
>
> i suggest you track things based on a combination of svn base url (ie:
> trunk vs a branch) and the specific svn revision number at the moment of
> your latest checkout -- that way you don't even need to subscribe to the
> commit list, just do an "svn diff -r" whenever you have some time to work
> on it and see what's been committed since the last time you worked on it.
>
> Hell: you could probably script all of this and have hudson do it for
> you.
>
>
> -Hoss
What we need is a way to map an SVN commit in Java Lucene to a Lucene.Net
JIRA issue. This way, we can track and assign those JIRA issues as port
tasks in Lucene.Net world.
I don't know much about how SVN revision numbers work. If from SVN revision
numbers, we can get back each commit, then this is the best solution
(especially if hudson can do it for us).
-- George
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Porting Java Lucene 2.9 to Lucene.Net (was: RE: Lucene 2.9 RC2
now available for testing)
Posted by Chris Hostetter <ho...@fucit.org>.
: My question is, I would prefer to track SVN commits to keep track of
: changes, vs. what I'm doing now. This will allow us to stay weeks
: behind a Java release vs. months or years as it is now. However, while
: I'm subscribed to SVN's commits mailing list, I'm not getting all those
: commits! For example, a commit made this past Friday, I never got an
: email for, while other commits I do. Any idea what maybe going on?
i suggest you track things based on a combination of svn base url (ie:
trunk vs a branch) and the specific svn revision number at the moment of
your latest checkout -- that way you don't even need to subscribe to the
commit list, just do an "svn diff -r" whenever you have some time to work
on it and see what's been committed since the last time you worked on it.
Hell: you could probably script all of this and have hudson do it for
you.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: Porting Java Lucene 2.9 to Lucene.Net
Posted by Uwe Schindler <uw...@thetaphi.de>.
I missed no commit mails the last days. Maybe they were declared as spam by
your mail filter software?
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: George Aroush [mailto:george@aroush.net]
> Sent: Saturday, August 29, 2009 10:36 PM
> To: java-dev@lucene.apache.org
> Subject: RE: Porting Java Lucene 2.9 to Lucene.Net
>
> I have been watching SVN commits for about 2-3 weeks now (since the
> announcement that the trunk was going to be an RC) during this time, I
> have
> not received any email. It was emails from JIRA that hinted me at code
> checkin's.
>
> -- George
>
> -----Original Message-----
> From: Mark Miller [mailto:markrmiller@gmail.com]
> Sent: Saturday, August 29, 2009 11:56 AM
> To: java-dev@lucene.apache.org
> Subject: Re: Porting Java Lucene 2.9 to Lucene.Net
>
> George Aroush wrote:
> >>> while I'm subscribed to SVN's commits mailing list, I'm not getting
> >>> all those commits! For example, a commit made this past Friday, I
> >>> never got an email for, while other commits I do. Any idea what
> >>> maybe going on?
> >>>
> >> There has been some flakiness with ASF servers, so maybe that is the
> >> reason? Are you using forwarding on people.a.o?
> >>
> >
> > No, I'm not using any forwarding.
> >
> > If the code, in the truck, goes through minimal changes for about 2
> weeks,
> I
> > should be fine (until when we figure out the mail issue).
> >
> > -- George
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-dev-help@lucene.apache.org
> >
> >
> Have you noticed it happen more than the one time? I've seen similar odd
> drops on the reg mailing list rarely - a random reply will show up to an
> original email I never got. Didn't notice it happen enough to claim its
> anything but bad luck though ;) It makes me wonder what I've missed and
> not realized though. I'm not forwarding either.
>
> bq. If the code, in the truck, goes through minimal changes for about 2
> weeks,
>
> It should be fairly stable over the next week - bug fixes, build issues,
> and
> doc are still fair game, but it should be quite contained and rarely
> involve
> code changes (knock on wood). When we unfreeze, there is likely to be
> rapid
> development moving towards 3.0 - we will branch off 2.9 before that
> though,
> and you will probably want to move from trunk to the branch.
>
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: Porting Java Lucene 2.9 to Lucene.Net
Posted by George Aroush <ge...@aroush.net>.
I have been watching SVN commits for about 2-3 weeks now (since the
announcement that the trunk was going to be an RC) during this time, I have
not received any email. It was emails from JIRA that hinted me at code
checkin's.
-- George
-----Original Message-----
From: Mark Miller [mailto:markrmiller@gmail.com]
Sent: Saturday, August 29, 2009 11:56 AM
To: java-dev@lucene.apache.org
Subject: Re: Porting Java Lucene 2.9 to Lucene.Net
George Aroush wrote:
>>> while I'm subscribed to SVN's commits mailing list, I'm not getting
>>> all those commits! For example, a commit made this past Friday, I
>>> never got an email for, while other commits I do. Any idea what
>>> maybe going on?
>>>
>> There has been some flakiness with ASF servers, so maybe that is the
>> reason? Are you using forwarding on people.a.o?
>>
>
> No, I'm not using any forwarding.
>
> If the code, in the truck, goes through minimal changes for about 2 weeks,
I
> should be fine (until when we figure out the mail issue).
>
> -- George
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
Have you noticed it happen more than the one time? I've seen similar odd
drops on the reg mailing list rarely - a random reply will show up to an
original email I never got. Didn't notice it happen enough to claim its
anything but bad luck though ;) It makes me wonder what I've missed and
not realized though. I'm not forwarding either.
bq. If the code, in the truck, goes through minimal changes for about 2
weeks,
It should be fairly stable over the next week - bug fixes, build issues, and
doc are still fair game, but it should be quite contained and rarely involve
code changes (knock on wood). When we unfreeze, there is likely to be rapid
development moving towards 3.0 - we will branch off 2.9 before that though,
and you will probably want to move from trunk to the branch.
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Porting Java Lucene 2.9 to Lucene.Net
Posted by Mark Miller <ma...@gmail.com>.
George Aroush wrote:
>>> while I'm subscribed to SVN's commits mailing list, I'm not getting
>>> all those commits! For example, a commit made this past Friday, I
>>> never got an email for, while other commits I do. Any idea what
>>> maybe going on?
>>>
>> There has been some flakiness with ASF servers, so maybe that is the
>> reason? Are you using forwarding on people.a.o?
>>
>
> No, I'm not using any forwarding.
>
> If the code, in the truck, goes through minimal changes for about 2 weeks, I
> should be fine (until when we figure out the mail issue).
>
> -- George
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
Have you noticed it happen more than the one time? I've seen similar odd
drops on the reg mailing list rarely - a random reply will show up to an
original email I never got. Didn't notice it happen enough to claim its
anything but bad luck though ;) It makes me wonder what I've missed and
not realized though. I'm not forwarding either.
bq. If the code, in the truck, goes through minimal changes for about 2 weeks,
It should be fairly stable over the next week - bug fixes, build issues, and doc are still fair game, but it should be quite contained and rarely involve code changes (knock on wood). When we unfreeze, there is likely to be rapid development moving towards 3.0 - we will branch off 2.9 before that though, and you will probably want to move from trunk to the branch.
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: Porting Java Lucene 2.9 to Lucene.Net (was: RE: Lucene 2.9 RC2 now available for testing)
Posted by George Aroush <ge...@aroush.net>.
> > while I'm subscribed to SVN's commits mailing list, I'm not getting
> > all those commits! For example, a commit made this past Friday, I
> > never got an email for, while other commits I do. Any idea what
> > maybe going on?
>
> There has been some flakiness with ASF servers, so maybe that is the
> reason? Are you using forwarding on people.a.o?
No, I'm not using any forwarding.
If the code, in the truck, goes through minimal changes for about 2 weeks, I
should be fine (until when we figure out the mail issue).
-- George
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Porting Java Lucene 2.9 to Lucene.Net (was: RE: Lucene 2.9 RC2 now available for testing)
Posted by Grant Ingersoll <gs...@apache.org>.
On Aug 29, 2009, at 10:58 AM, George Aroush wrote:
> Hi Folks,
>
> I have been following the expected release of Lucene.Net 2.9 for
> some weeks now, and about 2 weeks ago, just before RC1 was released,
> I grabbed the code off the trunk and started my initial port.
> Today, I compared what I have with what's in RC2, only 6 Java files
> changes -- this is good.
>
> My question is, I would prefer to track SVN commits to keep track of
> changes, vs. what I'm doing now. This will allow us to stay weeks
> behind a Java release vs. months or years as it is now. However,
> while I'm subscribed to SVN's commits mailing list, I'm not getting
> all those commits! For example, a commit made this past Friday, I
> never got an email for, while other commits I do. Any idea what
> maybe going on?
There has been some flakiness with ASF servers, so maybe that is the
reason? Are you using forwarding on people.a.o?
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Porting Java Lucene 2.9 to Lucene.Net (was: RE: Lucene 2.9 RC2 now available for testing)
Posted by George Aroush <ge...@aroush.net>.
Hi Folks,
I have been following the expected release of Lucene.Net 2.9 for some weeks now, and about 2 weeks ago, just before RC1 was released, I grabbed the code off the trunk and started my initial port. Today, I compared what I have with what's in RC2, only 6 Java files changes -- this is good.
My question is, I would prefer to track SVN commits to keep track of changes, vs. what I'm doing now. This will allow us to stay weeks behind a Java release vs. months or years as it is now. However, while I'm subscribed to SVN's commits mailing list, I'm not getting all those commits! For example, a commit made this past Friday, I never got an email for, while other commits I do. Any idea what maybe going on?
I just sent another subscription request email to SVN commit, but I'm not sure if that will make any difference.
Thanks.
-- George
-----Original Message-----
From: Mark Miller [mailto:markrmiller@gmail.com]
Sent: Friday, August 28, 2009 3:19 PM
To: java-user@lucene.apache.org
Subject: Re: Lucene 2.9 RC2 now available for testing
Mark Miller wrote:
>
> Download release candidate 1 here:
> http://people.apache.org/~markrmiller/staging-area/lucene2.9rc2/
>
In case anyone catches - yes that is a cut and paste typo - should read
release candidate 2 (obvious, but just to cross my t's).
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene 2.9 RC2 now available for testing
Posted by Mark Miller <ma...@gmail.com>.
Mark Miller wrote:
>
> Download release candidate 1 here:
> http://people.apache.org/~markrmiller/staging-area/lucene2.9rc2/
>
In case anyone catches - yes that is a cut and paste typo - should read
release candidate 2 (obvious, but just to cross my t's).
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org