No response from Solr on complex request (real issue explained)

[Xavier Schepler <xa...@sciences-po.fr>]

Hi,

We are in a beta testing phase, with several users a day.

After several days of running well, the solr server stopped responding 
to requests that require a lot of processing time, like this one :

wt=json&omitHeader=true&q=qiAndMSwFR%3A%28transport%29&q.op=AND&start=0&rows=5&fl=id,domainId,solrLangCode,ddiFileId,studyDescriptionId,studyYearAndDescriptionId,nesstarServerId,studyNesstarId,variableId,questionId,variableNesstarId,concept,studyTitle,studyQuestionCount,hasMultipleItems,variableName,hasQuestionnaire,questionnaireUrl,studyDescriptionUrl,universe,notes,preQuestionText,postQuestionText,interviewerInstructions,questionPosition,vlFR,qFR,iFR,mFR,vlEN,qEN,iEN,mEN,&sort=score%20desc&fq=solrLangCode%3AFR&facet=true&facet.field=%7B%21ex%3DstudySerieIds%2Cdecades%2CstudyIds%2CqueryFilters%2CconceptIds%2CdomainIds%7DdomainId&facet.field=%7B%21ex%3DstudySerieIds%2Cdecades%2CstudyIds%2CqueryFilters%2CconceptIds%2CdomainIds%7DstudyDecade&facet.field=%7B%21ex%3DstudySerieIds%2Cdecades%2CstudyIds%2CqueryFilters%2CconceptIds%2CdomainIds%7DstudySerieId&facet.field=%7B%21ex%3DstudySerieIds%2Cdecades%2CstudyIds%2CqueryFilters%2CconceptIds%2CdomainIds%7DstudyYearAndDescriptionId&facet.sort=count&f.studyDecade.facet.sort=lex&spellcheck=true&spellcheck.count=10&spellcheck.dictionary=qiAndMFR&spellcheck.q=transport&hl=on&hl.fl=qSwFR,iHLSwFR,mHLSwFR&hl.fragsize=0&hl.snippets=10000&hl.usePhraseHighlighter=true&hl.highlightMultiTerm=true&hl.simple.pre=%3Cb%3E&hl.simple.post=%3C%2Fb%3E&hl.mergeContiguous=false 


It involves highlighting on a multivalued field with more than 600 short 
values inside. Usually, it takes 200 or 300 ms.

I'm using Solr within Tomcat.
After restarting Tomcat all went fine again.

I'm trying to understand why I had to restart tomcat and what should I 
do to have it working 7/7 24/24.


Xavier

Re: No response from Solr on complex request (real issue explained)

[Erick Erickson <er...@gmail.com>]

This smells like a garbage collection issue. Here's an article from about a
year ago
by Mark Miller that might help:

http://www.lucidimagination.com/blog/2009/09/19/java-garbage-collection-boot-camp-draft/

<http://www.lucidimagination.com/blog/2009/09/19/java-garbage-collection-boot-camp-draft/>Does
the query #ever# come back or do you get impatient and kill it?

Also, what version of Solr are you using? Java? What op system? If it is
a garbage collection issue, the more details you can provide the better...

Best
Erick

On Thu, Oct 28, 2010 at 7:46 AM, Xavier Schepler <
xavier.schepler@sciences-po.fr> wrote:

> Hi,
>
> We are in a beta testing phase, with several users a day.
>
> After several days of running well, the solr server stopped responding to
> requests that require a lot of processing time, like this one :
>
> wt=json&omitHeader=true&q=qiAndMSwFR%3A%28transport%29&q.op=AND&start=0&rows=5&fl=id,domainId,solrLangCode,ddiFileId,studyDescriptionId,studyYearAndDescriptionId,nesstarServerId,studyNesstarId,variableId,questionId,variableNesstarId,concept,studyTitle,studyQuestionCount,hasMultipleItems,variableName,hasQuestionnaire,questionnaireUrl,studyDescriptionUrl,universe,notes,preQuestionText,postQuestionText,interviewerInstructions,questionPosition,vlFR,qFR,iFR,mFR,vlEN,qEN,iEN,mEN,&sort=score%20desc&fq=solrLangCode%3AFR&facet=true&facet.field=%7B%21ex%3DstudySerieIds%2Cdecades%2CstudyIds%2CqueryFilters%2CconceptIds%2CdomainIds%7DdomainId&facet.field=%7B%21ex%3DstudySerieIds%2Cdecades%2CstudyIds%2CqueryFilters%2CconceptIds%2CdomainIds%7DstudyDecade&facet.field=%7B%21ex%3DstudySerieIds%2Cdecades%2CstudyIds%2CqueryFilters%2CconceptIds%2CdomainIds%7DstudySerieId&facet.field=%7B%21ex%3DstudySerieIds%2Cdecades%2CstudyIds%2CqueryFilters%2CconceptIds%2CdomainIds%7DstudyYearAndDescriptionId&facet.sort=count&f.studyDecade.facet.sort=lex&spellcheck=true&spellcheck.count=10&spellcheck.dictionary=qiAndMFR&spellcheck.q=transport&hl=on&hl.fl=qSwFR,iHLSwFR,mHLSwFR&hl.fragsize=0&hl.snippets=10000&hl.usePhraseHighlighter=true&hl.highlightMultiTerm=true&hl.simple.pre=%3Cb%3E&hl.simple.post=%3C%2Fb%3E&hl.mergeContiguous=false
>
>
> It involves highlighting on a multivalued field with more than 600 short
> values inside. Usually, it takes 200 or 300 ms.
>
> I'm using Solr within Tomcat.
> After restarting Tomcat all went fine again.
>
> I'm trying to understand why I had to restart tomcat and what should I do
> to have it working 7/7 24/24.
>
>
> Xavier
>
>