You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@lucenenet.apache.org by Mark Burks <bi...@gmail.com> on 2008/01/10 05:19:46 UTC
Search while sorting is slow
I have an index that when i search for term with out sorting the results are
returned in .7 seconds or less. When I choose to search with a column chosen
for sorting the results will return in 7 - 10 seconds. The larger the # of
hits the longer it takes when sorting. The fields that can be sorted on are
unique to the document,stored in the index as un_tokenized. I am using
lucene.net 2.0. Index size is normally around 2.5 gig but i have the
same problem on a 500 meg index too. Has anyone else experienced this?
Any help will be greatly appreciated.
Thanks,
Mark Burks
Re: Search while sorting is slow
Posted by Mark Burks <bi...@gmail.com>.
After doing some digging into our search service I see that the group we
outsourced this project to wrote a custom hit collector and a separate
sorting method. This is probably due to the requirements of being able to
sort by up to 2 fields with either field being ascending or descending. I
see it is the sorting method they wrote is the problem. Thanks for every
one's input. Sorry I didn't research it further before I jumped to the
conclusion it was a lucene issue.
Mark
On Jan 12, 2008 6:26 AM, DIGY <di...@gmail.com> wrote:
> Lucene uses cached values for sorting and warm-up time of the cache takes
> nearly 99% of the search time. So, If you do not close the IndexSearcher
> other searches will be very fast.
>
> DIGY
>
> -----Original Message-----
> From: Mark Burks [mailto:birdsonbat@gmail.com]
> Sent: Thursday, January 10, 2008 2:36 PM
> To: lucene-net-user@incubator.apache.org
> Subject: Re: Search while sorting is slow
>
> Thanks Michael. I have considered doing as you have suggested but I was
> hoping that i was doing something wrong with the index build and it would
> be
> a quick fix. Guess I will be going down that road now.
>
> I don't understand why lucene is slow at sorting when that is one of
> things
> that it is supposed to do efficiently.
> Mark
>
> On Jan 10, 2008 12:04 AM, Michael Mitiaguin <mi...@gmail.com> wrote:
>
> > I also found built-in sorting very slow and ended up with populating
> > certain C# collection from hits in default order and then sorting that
> > collection. Apart from greater speed , more possibilities to
> > implement any kind of sorting.
> >
> > On Jan 10, 2008 3:19 PM, Mark Burks <bi...@gmail.com> wrote:
> > > I have an index that when i search for term with out sorting the
> results
> > are
> > > returned in .7 seconds or less. When I choose to search with a column
> > chosen
> > > for sorting the results will return in 7 - 10 seconds. The larger the
> #
> > of
> > > hits the longer it takes when sorting. The fields that can be sorted
> on
> > are
> > > unique to the document,stored in the index as un_tokenized. I am using
> > > lucene.net 2.0. Index size is normally around 2.5 gig but i have the
> > > same problem on a 500 meg index too. Has anyone else experienced
> this?
> > >
> > > Any help will be greatly appreciated.
> > >
> > > Thanks,
> > > Mark Burks
> > >
> >
>
>
RE: Search while sorting is slow
Posted by DIGY <di...@gmail.com>.
Lucene uses cached values for sorting and warm-up time of the cache takes
nearly 99% of the search time. So, If you do not close the IndexSearcher
other searches will be very fast.
DIGY
-----Original Message-----
From: Mark Burks [mailto:birdsonbat@gmail.com]
Sent: Thursday, January 10, 2008 2:36 PM
To: lucene-net-user@incubator.apache.org
Subject: Re: Search while sorting is slow
Thanks Michael. I have considered doing as you have suggested but I was
hoping that i was doing something wrong with the index build and it would be
a quick fix. Guess I will be going down that road now.
I don't understand why lucene is slow at sorting when that is one of things
that it is supposed to do efficiently.
Mark
On Jan 10, 2008 12:04 AM, Michael Mitiaguin <mi...@gmail.com> wrote:
> I also found built-in sorting very slow and ended up with populating
> certain C# collection from hits in default order and then sorting that
> collection. Apart from greater speed , more possibilities to
> implement any kind of sorting.
>
> On Jan 10, 2008 3:19 PM, Mark Burks <bi...@gmail.com> wrote:
> > I have an index that when i search for term with out sorting the results
> are
> > returned in .7 seconds or less. When I choose to search with a column
> chosen
> > for sorting the results will return in 7 - 10 seconds. The larger the #
> of
> > hits the longer it takes when sorting. The fields that can be sorted on
> are
> > unique to the document,stored in the index as un_tokenized. I am using
> > lucene.net 2.0. Index size is normally around 2.5 gig but i have the
> > same problem on a 500 meg index too. Has anyone else experienced this?
> >
> > Any help will be greatly appreciated.
> >
> > Thanks,
> > Mark Burks
> >
>
Re: Search while sorting is slow
Posted by Mark Burks <bi...@gmail.com>.
Thanks Michael. I have considered doing as you have suggested but I was
hoping that i was doing something wrong with the index build and it would be
a quick fix. Guess I will be going down that road now.
I don't understand why lucene is slow at sorting when that is one of things
that it is supposed to do efficiently.
Mark
On Jan 10, 2008 12:04 AM, Michael Mitiaguin <mi...@gmail.com> wrote:
> I also found built-in sorting very slow and ended up with populating
> certain C# collection from hits in default order and then sorting that
> collection. Apart from greater speed , more possibilities to
> implement any kind of sorting.
>
> On Jan 10, 2008 3:19 PM, Mark Burks <bi...@gmail.com> wrote:
> > I have an index that when i search for term with out sorting the results
> are
> > returned in .7 seconds or less. When I choose to search with a column
> chosen
> > for sorting the results will return in 7 - 10 seconds. The larger the #
> of
> > hits the longer it takes when sorting. The fields that can be sorted on
> are
> > unique to the document,stored in the index as un_tokenized. I am using
> > lucene.net 2.0. Index size is normally around 2.5 gig but i have the
> > same problem on a 500 meg index too. Has anyone else experienced this?
> >
> > Any help will be greatly appreciated.
> >
> > Thanks,
> > Mark Burks
> >
>
Re: Search while sorting is slow
Posted by Michael Mitiaguin <mi...@gmail.com>.
I also found built-in sorting very slow and ended up with populating
certain C# collection from hits in default order and then sorting that
collection. Apart from greater speed , more possibilities to
implement any kind of sorting.
On Jan 10, 2008 3:19 PM, Mark Burks <bi...@gmail.com> wrote:
> I have an index that when i search for term with out sorting the results are
> returned in .7 seconds or less. When I choose to search with a column chosen
> for sorting the results will return in 7 - 10 seconds. The larger the # of
> hits the longer it takes when sorting. The fields that can be sorted on are
> unique to the document,stored in the index as un_tokenized. I am using
> lucene.net 2.0. Index size is normally around 2.5 gig but i have the
> same problem on a 500 meg index too. Has anyone else experienced this?
>
> Any help will be greatly appreciated.
>
> Thanks,
> Mark Burks
>