Re: Solrj performance bottleneck

[rahul <as...@gmail.com>]

Hi All,

I just to want to share some findings which clearly identified the reason
for our performance bottleneck. we had looked into several areas for
optimization mostly directed at Solr configurations, stored fields,
highlighting, JVM, OS cache etc. But it turned out that the "main" culprit
was elsewhere. We were using the terms component for auto suggestion and
while examining the firebug outputs for time taken during the searches, we
detected that multiple requests were being spawned for autosuggestion as we
typed in the keyword to search (1 request per each character typed) and this
in turn cost us great delay in getting the search results. Once we turned
auto suggestion off, the performance was remarkably better and came down to
a second or so (compared to 8-10 seconds registered earlier).

if anybody has some suggestions/experience on how to leverage autosuggestion
without affecting search performance much, please do share them.

Once again, thanks for your inputs in analyzing our issues.

Thanks,

--
View this message in context: http://lucene.472066.n3.nabble.com/Solrj-performance-bottleneck-tp2682797p2775245.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solrj performance bottleneck

[Lance Norskog <go...@gmail.com>]

There is a separate auto-suggest tool that creates a simple in-memory
database outside of the Lucene index. This is called TST.

On Tue, Apr 5, 2011 at 3:36 AM, rahul <as...@gmail.com> wrote:
> Thanks Stefan and Victor ! we are using GWT for front end. We stopped issuing
> multiple asynchronous queries and issue a request and fetch results and then
> filter the results based on what has
> been typed subsequent to the request and then re trigger the request only if
> we don't get the expected results.
>
> Thanks Victor, I appreciate the link to the Jquery example and we will look
> into it as a reference.
>
> Regards,
> Rahul.
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Solrj-performance-bottleneck-tp2682797p2779387.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Lance Norskog
goksron@gmail.com

Re: Solrj performance bottleneck

[rahul <as...@gmail.com>]

Thanks Stefan and Victor ! we are using GWT for front end. We stopped issuing
multiple asynchronous queries and issue a request and fetch results and then
filter the results based on what has
been typed subsequent to the request and then re trigger the request only if
we don't get the expected results.

Thanks Victor, I appreciate the link to the Jquery example and we will look
into it as a reference.

Regards,
Rahul.

--
View this message in context: http://lucene.472066.n3.nabble.com/Solrj-performance-bottleneck-tp2682797p2779387.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solrj performance bottleneck

[openvictor Open <op...@gmail.com>]

Dear Rahul,

Stefan has the right solution. the autosuggest must be checked both from
Javascript and your backend. For javascript there are some really nice tools
to do that such as Jquery which implements a auto-suggest with a tunable
delay. It has also highlighting, you can add additional information etc...
It is actually quite impressive. Here is the address :
http://jqueryui.com/demos/autocomplete/#remote-jsonp. It's open source so
you can just copy what they have done or see the method they used.
For backend limit the number of request / second per ip or session and / or
cache result. As for cache normally solr caches the common request but I
don't know for term components.

Hope this helps you !

Victor

2011/4/4 Stefan Matheis <ma...@googlemail.com>

> rahul,
>
> On Mon, Apr 4, 2011 at 4:18 PM, rahul <as...@gmail.com> wrote:
> > if anybody has some suggestions/experience on how to leverage
> autosuggestion
> > without affecting search performance much, please do share them.
>
> we use javascript intervals for autosuggestion. regularly check the
> value of the monitored input field and if changed, trigger a new
> request. this will cover both cases, slow-typing users and also
> ten-finger-guys (which will type much faster). a new request for every
> added character is indeed too much, even if your backend is responding
> within a few ms.
>
> Regards
> Stefan
>

Re: Solrj performance bottleneck

[Stefan Matheis <ma...@googlemail.com>]

rahul,

On Mon, Apr 4, 2011 at 4:18 PM, rahul <as...@gmail.com> wrote:
> if anybody has some suggestions/experience on how to leverage autosuggestion
> without affecting search performance much, please do share them.

we use javascript intervals for autosuggestion. regularly check the
value of the monitored input field and if changed, trigger a new
request. this will cover both cases, slow-typing users and also
ten-finger-guys (which will type much faster). a new request for every
added character is indeed too much, even if your backend is responding
within a few ms.

Regards
Stefan