Re[4]: how to enhance speed of sorted search

[Yura Smolsky <in...@altervisionmedia.com>]

Hello, Chris.

CH> : I am thinking should be this faster
CH> The ConstantScoreQuery wrapped arround the QueryFilter might in fact be
CH> faster then the raw query -- have your tried it to see?

I will try when PrefixFilter will be added in Lucene. b/c I have some
technical issues and I use PyLucene. But on my local index of 400 mb is
it not slower than usual :)

CH> you might be able to shave a little bit of speed off by accessing the bits
CH> from the Filter directly and iterating over them yourself to check the
CH> FieldCache ad build up your sorted list of the first "N" -- i think that
CH> would save you one method call per match (the score method of
CH> ConstantScoreQuery)

What is the "sorted list of the first "N""?

CH> At some point you just have to wonder if it's fast enough?

:)

CH> how long does a typically sorted query take for you right now?

it takes 0.2 up to 0.4. this is only searching without iterating over
hits and fetching docs.

CH> how many documents are in your index?

about 30 millions

CH> how many matches do you typically have?

it varies greatly. from 7k to 100k


--
Yura Smolsky,
http://altervisionmedia.com/



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re[4]: how to enhance speed of sorted search

[Chris Hostetter <ho...@fucit.org>]

: CH> you might be able to shave a little bit of speed off by accessing the bits
: CH> from the Filter directly and iterating over them yourself to check the
: CH> FieldCache ad build up your sorted list of the first "N" -- i think that
: CH> would save you one method call per match (the score method of
: CH> ConstantScoreQuery)
:
: What is the "sorted list of the first "N""?

Whatever you want it to be ... my point is merely that the existing Lucene
code for sorted searching involves a "Query" -- if you know you want to
start with a Filter, then you might be able to come up with a faster
mechanism for getting sorted results using the BitSet directly, collecting
however many of the sorted resutls you care about in a data structure of
your choosing.

but this is all just speculation.



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org