Search timeout for Solrcloud

[arin_g <ar...@gmail.com>]

Hi, 
We use solrcloud in production, and we are facing some issues with queries
that take very long specially deep paging queries, these queries keep our
servers very busy. i am looking for a way to stop (kill) queries taking
longer than a specific amount of time (say 5 seconds), i checked timeAllowed
but it doesn't work (again query  runs completely). Also i noticed that
there are connTimeout and socketTimeout for distributed searches, but i am
not sure if they kill the thread (i want to save resources by killing the
query, not just returning a timeout). Also, if i could get partial results
that would be ideal. Any suggestions?

Thanks,
 arin

--
View this message in context: http://lucene.472066.n3.nabble.com/Search-timeout-for-Solrcloud-tp3987716.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Search timeout for Solrcloud

[arin_g <ar...@gmail.com>]

for example when we set the start parameter to 1000, 2000 or higher (page
100, 200 ...), it takes very long (20, 30 seconds, sometimes even 100
seconds).
this usually happens when there is a big gap between pages, mostly hit by
web crawlers (when they crawl the last page link on our website).


--
View this message in context: http://lucene.472066.n3.nabble.com/Search-timeout-for-Solrcloud-tp3987716p3987834.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Search timeout for Solrcloud

[Jack Krupansky <ja...@basetechnology.com>]

I'm curious... how deep is it that is becoming problematic? Tens of pages, 
hundreds, thousands, millions?

And when you say deep paging, are you incrementing through all pages down to 
the depth or "gapping" to some very large depth outright? If the former, I 
am wondering if the Solr cache is building up with all those previous 
results.

And is it that the time is simply moderately beyond expectations (e.g. 10 or 
30 seconds or a minute compared to 1 second), or... are we talking about a 
situation where a core is terminally "thrashing" with garbage collection/OOM 
issues?

-- Jack Krupansky

-----Original Message----- 
From: arin_g
Sent: Tuesday, June 05, 2012 1:34 AM
To: solr-user@lucene.apache.org
Subject: Search timeout for Solrcloud

Hi,
We use solrcloud in production, and we are facing some issues with queries
that take very long specially deep paging queries, these queries keep our
servers very busy. i am looking for a way to stop (kill) queries taking
longer than a specific amount of time (say 5 seconds), i checked timeAllowed
but it doesn't work (again query  runs completely). Also i noticed that
there are connTimeout and socketTimeout for distributed searches, but i am
not sure if they kill the thread (i want to save resources by killing the
query, not just returning a timeout). Also, if i could get partial results
that would be ideal. Any suggestions?

Thanks,
arin

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Search-timeout-for-Solrcloud-tp3987716.html
Sent from the Solr - User mailing list archive at Nabble.com. 

Re: Search timeout for Solrcloud

[Jason Rutherglen <ja...@gmail.com>]

There isn't a solution for killing long running queries that works.

On Tue, Jun 5, 2012 at 1:34 AM, arin_g <ar...@gmail.com> wrote:
> Hi,
> We use solrcloud in production, and we are facing some issues with queries
> that take very long specially deep paging queries, these queries keep our
> servers very busy. i am looking for a way to stop (kill) queries taking
> longer than a specific amount of time (say 5 seconds), i checked timeAllowed
> but it doesn't work (again query  runs completely). Also i noticed that
> there are connTimeout and socketTimeout for distributed searches, but i am
> not sure if they kill the thread (i want to save resources by killing the
> query, not just returning a timeout). Also, if i could get partial results
> that would be ideal. Any suggestions?
>
> Thanks,
>  arin
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Search-timeout-for-Solrcloud-tp3987716.html
> Sent from the Solr - User mailing list archive at Nabble.com.

RE: Search timeout for Solrcloud

[Markus Jelsma <ma...@openindex.io>]

There's an open issue for improving deep paging performance:
https://issues.apache.org/jira/browse/SOLR-1726
 
 
-----Original message-----
> From:arin_g <ar...@gmail.com>
> Sent: Tue 05-Jun-2012 12:03
> To: solr-user@lucene.apache.org
> Subject: Search timeout for Solrcloud
> 
> Hi, 
> We use solrcloud in production, and we are facing some issues with queries
> that take very long specially deep paging queries, these queries keep our
> servers very busy. i am looking for a way to stop (kill) queries taking
> longer than a specific amount of time (say 5 seconds), i checked timeAllowed
> but it doesn't work (again query  runs completely). Also i noticed that
> there are connTimeout and socketTimeout for distributed searches, but i am
> not sure if they kill the thread (i want to save resources by killing the
> query, not just returning a timeout). Also, if i could get partial results
> that would be ideal. Any suggestions?
> 
> Thanks,
>  arin
> 
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Search-timeout-for-Solrcloud-tp3987716.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>