You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Leos Literak <li...@seznam.cz> on 2006/01/07 16:10:43 UTC
how to forbid prefetching found Documents?
Hello,
I found in Lucenebook, that searching always reads
first first 100 documents. I take a look to java
source code and found that in Hits.java. I cannot
find any easy way, how to forbid this behaviour.
motivation: my page size is 30. It is useless to
read 100 documents, when I will display only small
portion of it. And imagine, that user is on fourth
page - reading first 100 document is waste of time.
Thanks for your idea
Leos
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: how to forbid prefetching found Documents?
Posted by Chris Hostetter <ho...@fucit.org>.
This is the reason i suggested a while back that the initiall call to to
getMoreDocs should be moved out of the Hits constructor, so no prefetching
is done untill the client asks for hte first hit they want -- if on page#2
of some paginated results we wants docs 101-200, then the first prefetch
will get 1-200
http://www.nabble.com/Weird-time-results-doing-wildcard-queries-t291393.html#a818816
...but since I never actaully use a Hits object in any of my apps, i
haven't tried actually making these changes to see if it works out well.
: > Yonik, I want to display 120th. up to 150th. document
: > in Hits. Do you mean that Hits does not contain id
: > of all relevant documents?
:
: Correct, it does not. The first time Hits is returned to you, it will
: internally contain the ids and scores of the top 100, but not the
: stored fields (the Document instance).
:
: When you access beyond the 100th element, Hits will internally redo
: the search asking for more ids and scores this time.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: how to forbid prefetching found Documents?
Posted by Leos Literak <li...@seznam.cz>.
Yonik Seeley wrote:
> On 1/7/06, Leos Literak <li...@seznam.cz> wrote:
>
>>Yonik, I want to display 120th. up to 150th. document
>>in Hits. Do you mean that Hits does not contain id
>>of all relevant documents?
>
>
> Correct, it does not. The first time Hits is returned to you, it will
> internally contain the ids and scores of the top 100, but not the
> stored fields (the Document instance).
>
> When you access beyond the 100th element, Hits will internally redo
> the search asking for more ids and scores this time.
thanks, I understand it now finally. Glad to hear that complete document
is not loaded at this time.
Leos
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: how to forbid prefetching found Documents?
Posted by Yonik Seeley <ys...@gmail.com>.
On 1/7/06, Leos Literak <li...@seznam.cz> wrote:
> Yonik, I want to display 120th. up to 150th. document
> in Hits. Do you mean that Hits does not contain id
> of all relevant documents?
Correct, it does not. The first time Hits is returned to you, it will
internally contain the ids and scores of the top 100, but not the
stored fields (the Document instance).
When you access beyond the 100th element, Hits will internally redo
the search asking for more ids and scores this time.
-Yonik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: how to forbid prefetching found Documents?
Posted by Leos Literak <li...@seznam.cz>.
Yonik, I want to display 120th. up to 150th. document
in Hits. Do you mean that Hits does not contain id
of all relevant documents? I thought that search will
return all document ids with their score and Lucene
prefetches first 100 document to speed the process.
My index contains complete text (yes, it is stored),
so feeding 100 documents, when I will display only 30
of them (or noone at all) is IMHO waste of time.
I would like to have a control and specify, how many
documents to prefetch.
Am I wrong?
leos
Yonik Seeley wrote:
> The actual fields of found documents are not prefetched, only the ids.
>
>
>>And imagine, that user is on fourth
>>page - reading first 100 document is waste of time.
>
>
> As it relates to document ids, you must know what the first 100 are if
> you are to know which ones follow.
>
> If you want more control over how this is all done, you can use the
> expert level search methods that return TopDocs or TopFieldDocs.
>
> -Yonik
>
>
> On 1/7/06, Leos Literak <li...@seznam.cz> wrote:
>
>>Hello,
>>
>>I found in Lucenebook, that searching always reads
>>first first 100 documents. I take a look to java
>>source code and found that in Hits.java. I cannot
>>find any easy way, how to forbid this behaviour.
>>
>>motivation: my page size is 30. It is useless to
>>read 100 documents, when I will display only small
>>portion of it. And imagine, that user is on fourth
>>page - reading first 100 document is waste of time.
>>
>>Thanks for your idea
>>
>>Leos
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: how to forbid prefetching found Documents?
Posted by Yonik Seeley <ys...@gmail.com>.
The actual fields of found documents are not prefetched, only the ids.
> And imagine, that user is on fourth
> page - reading first 100 document is waste of time.
As it relates to document ids, you must know what the first 100 are if
you are to know which ones follow.
If you want more control over how this is all done, you can use the
expert level search methods that return TopDocs or TopFieldDocs.
-Yonik
On 1/7/06, Leos Literak <li...@seznam.cz> wrote:
> Hello,
>
> I found in Lucenebook, that searching always reads
> first first 100 documents. I take a look to java
> source code and found that in Hits.java. I cannot
> find any easy way, how to forbid this behaviour.
>
> motivation: my page size is 30. It is useless to
> read 100 documents, when I will display only small
> portion of it. And imagine, that user is on fourth
> page - reading first 100 document is waste of time.
>
> Thanks for your idea
>
> Leos
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org