You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Brisbart Franck <Fr...@kelkoo.net> on 2003/04/07 14:59:33 UTC
Re: how to get an extra count
Ype Kingma wrote:
> On Tuesday 04 February 2003 09:12, you wrote:
>
>>Hi all,
>>
>>I'm trying to gather information about my non-searched (ie not used for
>>the search) fields.
>>Let's take an index with 2 fields: 'artist' (for the artist name) an
>>'type' (for his type of music).
>>I need to perform a search on the 'artist' field to retrieve a list of
>>artists matching the query. But I also need to have the number of
>>artists per 'type'.
>>My first solution was to write a HitCollector which do the work:
>> public void collect(int doc, float score) {
>> Document document = indexReader_.document(doc);
>> if(document.get("type").equals("Rock"))
>> nbRock++;
>> ...
>> }
>>But, as I first get the document to analyze my non-searched field, the
>>treatment can be very long for searches with lots of results.
>>
>>Is there any better(=faster) method to have this extra info ??
>
>
> It's best not to retrieve document fields during search.
> You can do this after collecting, possibly in the order
> that you collected the documents.
>
You're right. Using document fields during a search wasn't a good idea.
In fact, I would like to avoid using document fields at all. Because, it
takes too much time. If a search returns 10000 results, I need to
analyse all of them to get my extra info (nb of artist per type). And
even if I have only 10 results to display, my search is very slow.
It'll be great to treat my 'type' field the same way as the 'freq' is
used. I think this will be much faster.
How can I store the content of this field in a lucene file (such as
<segment>.frq) to use it during my search ??
thx,
Franck
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: how to get an extra count
Posted by Brisbart Franck <Fr...@kelkoo.net>.
Ype Kingma wrote:
> Franck,
>
> On Monday 07 April 2003 05:59, Brisbart Franck wrote:
>
>>Ype Kingma wrote:
>>
>>>On Tuesday 04 February 2003 09:12, you wrote:
>>
>
> <flashing back>
>
>>>>Hi all,
>>>>
>>>>I'm trying to gather information about my non-searched (ie not used for
>>>>the search) fields.
>>>>Let's take an index with 2 fields: 'artist' (for the artist name) an
>>>>'type' (for his type of music).
>>>>I need to perform a search on the 'artist' field to retrieve a list of
>>>>artists matching the query. But I also need to have the number of
>>>>artists per 'type'.
>>>>My first solution was to write a HitCollector which do the work:
>>>> public void collect(int doc, float score) {
>>>> Document document = indexReader_.document(doc);
>>>> if(document.get("type").equals("Rock"))
>>>> nbRock++;
>>>> ...
>>>> }
>>>>But, as I first get the document to analyze my non-searched field, the
>>>>treatment can be very long for searches with lots of results.
>>>>
>>>>Is there any better(=faster) method to have this extra info ??
>>>
>>>It's best not to retrieve document fields during search.
>>>You can do this after collecting, possibly in the order
>>>that you collected the documents.
>>
>>You're right. Using document fields during a search wasn't a good idea.
>>In fact, I would like to avoid using document fields at all. Because, it
>>takes too much time. If a search returns 10000 results, I need to
>>analyse all of them to get my extra info (nb of artist per type). And
>>even if I have only 10 results to display, my search is very slow.
>>It'll be great to treat my 'type' field the same way as the 'freq' is
>>used. I think this will be much faster.
>>How can I store the content of this field in a lucene file (such as
>><segment>.frq) to use it during my search ??
>
>
> Couldn't you extend your query with
>
> +type:rock
>
> and use the high level search API and ask for the size of the results?
> You can do that when you index the 'type' field.
>
If I do that, I should run 1 query for each 'type'. And, as I have
something like 2000 types, I don't think it is better to have 2000 fast
requests than 1 big request.
I could do 1 query for each type found by the first query. But it only
shifts the problem, because I should first retrieve all these types, and
to do that I have to analyse all my results.
My real problem is to find the best way to use extra data (not search
fields) to compute some meta data on a given query.
Franck
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: how to get an extra count
Posted by Ype Kingma <yk...@xs4all.nl>.
Franck,
On Monday 07 April 2003 05:59, Brisbart Franck wrote:
> Ype Kingma wrote:
> > On Tuesday 04 February 2003 09:12, you wrote:
<flashing back>
> >>Hi all,
> >>
> >>I'm trying to gather information about my non-searched (ie not used for
> >>the search) fields.
> >>Let's take an index with 2 fields: 'artist' (for the artist name) an
> >>'type' (for his type of music).
> >>I need to perform a search on the 'artist' field to retrieve a list of
> >>artists matching the query. But I also need to have the number of
> >>artists per 'type'.
> >>My first solution was to write a HitCollector which do the work:
> >> public void collect(int doc, float score) {
> >> Document document = indexReader_.document(doc);
> >> if(document.get("type").equals("Rock"))
> >> nbRock++;
> >> ...
> >> }
> >>But, as I first get the document to analyze my non-searched field, the
> >>treatment can be very long for searches with lots of results.
> >>
> >>Is there any better(=faster) method to have this extra info ??
> >
> > It's best not to retrieve document fields during search.
> > You can do this after collecting, possibly in the order
> > that you collected the documents.
>
> You're right. Using document fields during a search wasn't a good idea.
> In fact, I would like to avoid using document fields at all. Because, it
> takes too much time. If a search returns 10000 results, I need to
> analyse all of them to get my extra info (nb of artist per type). And
> even if I have only 10 results to display, my search is very slow.
> It'll be great to treat my 'type' field the same way as the 'freq' is
> used. I think this will be much faster.
> How can I store the content of this field in a lucene file (such as
> <segment>.frq) to use it during my search ??
Couldn't you extend your query with
+type:rock
and use the high level search API and ask for the size of the results?
You can do that when you index the 'type' field.
Kind regards,
Ype
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org