You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Yura Smolsky <in...@altervisionmedia.com> on 2006/09/21 18:12:35 UTC

Re[2]: ParallelMultiSearcher

Hello, Ronnie.

RK> Dont ask to ask, just ask! ;)

ok. I have big issue when I try to search ParallelMultiSearcher for
PrefixQuery. This query is being rewritten to BooleanQuery during
search. This causes Similarity to calculate docFreq for each Term in the
BooleanQuery. So if we have a lot of results for some PrefixQuery then
we have a lot of calls to docFreq method of Searchable object passed
to ParallelMultiSearcher. In my case this Searchable object exists on the
other computer (network). Search became very slow b/c
of those multiple calls of docFreq over net.

I am not sure if this question for users mail list. But I have spent
about 3 days to fix this problem and I do not see any solution.

Maybe developers of Lucene could suggest something...

Thanks and sorry for my bad English.

--
Yura Smolsky,
http://altervisionmedia.com/



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re[4]: ParallelMultiSearcher

Posted by Yura Smolsky <in...@altervisionmedia.com>.
Hello, Yonik.

>> ok. I have big issue when I try to search ParallelMultiSearcher for
>> PrefixQuery. This query is being rewritten to BooleanQuery during
>> search. This causes Similarity to calculate docFreq for each Term in the
>> BooleanQuery. So if we have a lot of results for some PrefixQuery then
>> we have a lot of calls to docFreq method of Searchable object passed
>> to ParallelMultiSearcher.

YS> IDF often does not make sense for auto-expanding queries (range, prefix, etc).
YS> If you don't need the idf factor that makes rarer terms count more,
YS> then use a PrefixFilter wrapped in a ConstantScoreQuery.

YS> http://lucene.apache.org/java/docs/api/org/apache/lucene/search/ConstantScoreQuery.html
YS> http://incubator.apache.org/solr/docs/api/org/apache/solr/search/PrefixFilter.html

Thank you so much! PrefixFilter has eliminated counting idf's.. I
wonder why it is not inside of Lucene yet :)

--
Yura Smolsky,
http://altervisionmedia.com/



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Re[2]: ParallelMultiSearcher

Posted by Yonik Seeley <yo...@apache.org>.
On 9/21/06, Yura Smolsky <in...@altervisionmedia.com> wrote:

> ok. I have big issue when I try to search ParallelMultiSearcher for
> PrefixQuery. This query is being rewritten to BooleanQuery during
> search. This causes Similarity to calculate docFreq for each Term in the
> BooleanQuery. So if we have a lot of results for some PrefixQuery then
> we have a lot of calls to docFreq method of Searchable object passed
> to ParallelMultiSearcher.

IDF often does not make sense for auto-expanding queries (range, prefix, etc).
If you don't need the idf factor that makes rarer terms count more,
then use a PrefixFilter wrapped in a ConstantScoreQuery.

http://lucene.apache.org/java/docs/api/org/apache/lucene/search/ConstantScoreQuery.html
http://incubator.apache.org/solr/docs/api/org/apache/solr/search/PrefixFilter.html

-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org