You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by harjags <ha...@bestbuy.com> on 2019/10/13 04:59:14 UTC
Solr 7.6 query performace question
We are upgrading to solr 7.6 from 6.1
Our query has below pattern predominantly
1.q is * as we filter based on a department of products always
2. 100+ bq's to boost certain document
3. Collapsing using a non DocValue field
4.Many Facet Fields and Many Facet queries
Which of the above is the most memory consuming operations?
Below errors are very common in 7.6 and we have solr nodes failing with
tanking memory.
The request took too long to iterate over terms. Timeout: timeoutAt:
162874656583645 (System.nanoTime(): 162874701942020),
TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@74507f4a
or
#*BitSetDocTopFilter*]; The request took too long to iterate over terms.
Timeout: timeoutAt: 33288640223586 (System.nanoTime(): 33288700895778),
TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@5e458644
or
#SortedIntDocSetTopFilter]; The request took too long to iterate over terms.
Timeout: timeoutAt: 552497919389297 (System.nanoTime(): 552508251053558),
TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@60b7186e
OR
--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Solr 7.6 query performace question
Posted by Erick Erickson <er...@gmail.com>.
Well, It Depends (tm).
Certainly 2 and 3 are _not_ memory intensive
4 depends on the number of terms in the fields.
But I suspect your real problem has nothing to do with memory and is <1>. Try q=*:* rather than q=*. In case your e-mail tries to make things bold, that’s q=asterisk-colon-asterisk, not q=asterisk
Best,
Erick
> On Oct 13, 2019, at 12:59 AM, harjags <ha...@bestbuy.com> wrote:
>
> We are upgrading to solr 7.6 from 6.1
> Our query has below pattern predominantly
>
> 1.q is * as we filter based on a department of products always
> 2. 100+ bq's to boost certain document
> 3. Collapsing using a non DocValue field
> 4.Many Facet Fields and Many Facet queries
>
> Which of the above is the most memory consuming operations?
>
> Below errors are very common in 7.6 and we have solr nodes failing with
> tanking memory.
>
> The request took too long to iterate over terms. Timeout: timeoutAt:
> 162874656583645 (System.nanoTime(): 162874701942020),
> TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@74507f4a
>
> or
>
> #*BitSetDocTopFilter*]; The request took too long to iterate over terms.
> Timeout: timeoutAt: 33288640223586 (System.nanoTime(): 33288700895778),
> TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@5e458644
>
>
> or
>
> #SortedIntDocSetTopFilter]; The request took too long to iterate over terms.
> Timeout: timeoutAt: 552497919389297 (System.nanoTime(): 552508251053558),
> TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@60b7186e
>
>
>
> OR
>
>
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Solr 7.6 query performace question
Posted by "raj.yadav" <ra...@cse.ism.ac.in>.
harjags wrote
> Below errors are very common in 7.6 and we have solr nodes failing with
> tanking memory.
>
> The request took too long to iterate over terms. Timeout: timeoutAt:
> 162874656583645 (System.nanoTime(): 162874701942020),
> TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@74507f4a
>
> or
>
> #*BitSetDocTopFilter*]; The request took too long to iterate over terms.
> Timeout: timeoutAt: 33288640223586 (System.nanoTime(): 33288700895778),
> TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@5e458644
>
>
> or
>
> #SortedIntDocSetTopFilter]; The request took too long to iterate over
> terms.
> Timeout: timeoutAt: 552497919389297 (System.nanoTime(): 552508251053558),
> TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@60b7186e
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
We are also seeing such errors in our log. But our nodes are not failing
also the frequency of such warnings are less then 5% of overall traffic.
What does this error means.
Can someone eleaborate following :
1. What does `The request took too long to iterate over terms` means ?
2. what is `BitSetDocTopFilter` and `SortedIntDocSetTopFilter` ?
Regards,
Raj
--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html