You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Qi Ouyang <al...@gmail.com> on 2010/12/17 05:43:25 UTC
Got error when range query and highlight
Hello all,
I got an error as follows when I do a range query search ([1 TO *])
on an numeric field and highlight is set on another text field.
2010/12/15 10:58:55 org.apache.solr.common.SolrException log
Fatal: org.apache.lucene.search.BooleanQuery$TooManyClauses:
maxClauseCount is set to 1024
at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:153)
at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:144)
at org.apache.lucene.search.MultiTermQuery$ScoringBooleanQueryRewrite.rewrite(MultiTermQuery.java:110)
at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:382)
at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:178)
at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:111)
at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:111)
at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.getWeightedSpanTerms(WeightedSpanTermExtractor.java:414) at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
at jp.co.spectrum.insight.hooserver.core.solrext.dispatcher.HooDispatchFilter.doFilter(Unknown
Source)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.lucene.search.highlight.QueryScorer.initExtractor(QueryScorer.java:216)
at org.apache.lucene.search.highlight.QueryScorer.init(QueryScorer.java:184)
at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:226)
at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:335)
at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:89)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at jp.co.spectrum.insight.hooserver.core.solrext.dispatcher.HooDispatchFilter.execute(Unknown
Source)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Unknown Source)
Could anyone give me any suggest?
Ouyang
https://sites.google.com/a/spectrum.co.jp/openinsight/
Re: Got error when range query and highlight
Posted by Erick Erickson <er...@gmail.com>.
Ahhh, ignore my blather, I need more coffee. Somehow I was thinking
about something else.
Best
Erick
On Fri, Dec 17, 2010 at 8:16 AM, Erick Erickson <er...@gmail.com>wrote:
> Let's back up a bit here. What is the field definition for the field you're
> doing the range query on? i.e. [1 TO *]? And what are you trying
> to accomplish with that clause (that is, all non-zero documents?
> all documents? Do you have counts < 1?).
>
> What version of Solr are you using?
>
> You'll also gain insights by stripping away all the other stuff and
> concentrating
> on one thing at a time. I suspect that you're using one of the field
> types that isn't very good with range queries and that query-time
> expansion is blowing the max boolean clauses limit. Try your
> query with &debugQuery=on and a smaller range, say [1 TO 10].
> I suspect you'll see 10 OR clauses.
>
> But I've been very wrong before <G>...
>
> Best
> Erick
>
> On Fri, Dec 17, 2010 at 3:06 AM, Qi Ouyang <al...@gmail.com> wrote:
>
>> Now I think that hl.requireFieldMatch=false is a problem, because
>> there is no meaning to match query string of "kcmeta/view/count" to
>> the high-light field "title" and "body". To support for full-text
>> query field search matching high-light fields, I think it's better to
>> add same query condition on "title" and "body".
>> Ex. original query content:"book"
>> convert to query : content:"book" OR title:"book" OR body:"book"
>> By using this query, I think it is possible to highlight "title" and
>> "body" by the query on "content".
>>
>> Thank for your suggestion.
>>
>> Ouyang
>>
>> https://sites.google.com/a/spectrum.co.jp/openinsight/
>>
>
>
Re: Got error when range query and highlight
Posted by Erick Erickson <er...@gmail.com>.
Let's back up a bit here. What is the field definition for the field you're
doing the range query on? i.e. [1 TO *]? And what are you trying
to accomplish with that clause (that is, all non-zero documents?
all documents? Do you have counts < 1?).
What version of Solr are you using?
You'll also gain insights by stripping away all the other stuff and
concentrating
on one thing at a time. I suspect that you're using one of the field
types that isn't very good with range queries and that query-time
expansion is blowing the max boolean clauses limit. Try your
query with &debugQuery=on and a smaller range, say [1 TO 10].
I suspect you'll see 10 OR clauses.
But I've been very wrong before <G>...
Best
Erick
On Fri, Dec 17, 2010 at 3:06 AM, Qi Ouyang <al...@gmail.com> wrote:
> Now I think that hl.requireFieldMatch=false is a problem, because
> there is no meaning to match query string of "kcmeta/view/count" to
> the high-light field "title" and "body". To support for full-text
> query field search matching high-light fields, I think it's better to
> add same query condition on "title" and "body".
> Ex. original query content:"book"
> convert to query : content:"book" OR title:"book" OR body:"book"
> By using this query, I think it is possible to highlight "title" and
> "body" by the query on "content".
>
> Thank for your suggestion.
>
> Ouyang
>
> https://sites.google.com/a/spectrum.co.jp/openinsight/
>
Re: Got error when range query and highlight
Posted by Qi Ouyang <al...@gmail.com>.
Now I think that hl.requireFieldMatch=false is a problem, because
there is no meaning to match query string of "kcmeta/view/count" to
the high-light field "title" and "body". To support for full-text
query field search matching high-light fields, I think it's better to
add same query condition on "title" and "body".
Ex. original query content:"book"
convert to query : content:"book" OR title:"book" OR body:"book"
By using this query, I think it is possible to highlight "title" and
"body" by the query on "content".
Thank for your suggestion.
Ouyang
https://sites.google.com/a/spectrum.co.jp/openinsight/
Re: Got error when range query and highlight
Posted by Ahmet Arslan <io...@yahoo.com>.
> > Adding &hl.requireFieldMatch=true should probably
> solve your problem.
> >
> Yes, adding &hl.requireFieldMatch=true can solve my
> problem, but in my
> solution , I have a "content" field indexing all fields'
> contents to
> support full text search, but I also have another 2 fields
> "title" and
> "body" which support highlight, when I do search on
> content, I expect
> the "title" and "body" can be high-lighted. So using the
> hl.requireFieldMatch=true may be not work.
So you can increase the number of max boolean clauses in solrconfig.xml. Default is 1024. Or you can use hl.highlightMultiTerm=false.
By the way I couldn't see full-text query in your URL. It is better to filter out your non full-text queries into filter queries. This can solve your problem. For example
&fq=cmeta/view/count:[ 1 TO * ]&fq=contentstype:A9000B0001*&fq=(issecure:"0" OR userid:"c6305dc4\-cbba\-bf48\-97d5\-dcfe6f2430ef")
You can define multiple fq's. And you can benefit caching.
http://wiki.apache.org/solr/CommonQueryParameters#fq
http://wiki.apache.org/solr/FilterQueryGuidance
Re: Got error when range query and highlight
Posted by Qi Ouyang <al...@gmail.com>.
Thank you for reply.
> Are you using hl.highlightMultiTerm=true? Pasting your search URL can give more hints.
>
Yes, I used the "hl.highlightMultiTerm=true" , my search query is as follows :
start=0&rows=10&facet.mincount=1&facet.field=authornav&facet.field=contentsearchkeywordnav&facet.field=contentstypenav&facet.field=copyrightnav&facet.field=docdatetimenav&facet.field=downloadpathnav&facet.field=filenamenav&facet.field=folderchecksumnav&facet.field=folderpathnav&facet.field=groupidnav&facet.field=kcmeta%2Fbookmark%2Fcountnav&facet.field=kcmeta%2Fcomment%2Fcountnav&facet.field=kcmeta%2Fenterprisetag%2Fcountnav&facet.field=kcmeta%2Fenterprisetag%2Fvaluenav&facet.field=kcmeta%2Fusertag%2Fcountnav&facet.field=kcmeta%2Fusertag%2Fvaluenav&facet.field=kcmeta%2Fview%2Fcountnav&facet.field=kcmeta%2Fvote%2Fcountnav&facet.field=lastmodifiernav&facet.field=mimetypenav&facet.field=orgidnav&facet.field=originalcontentstypenav&facet.field=processingtimenav&facet.field=roleidnav&facet.field=sizenav&facet.field=sourcenav&facet.field=titlenav&facet.field=useridnav&facet=true&hl=true&hl.fl=body&hl.fl=title&hl.simple.pre=%3Cspan+class%3D%22highlight-solr%22%3E&hl.simple.post=%3C%2Fspan%3E&q=%28%28kcmeta%2Fview%2Fcount%3A%5B+1+TO+*+%5D+AND+contentstype%3AA9000B0001*%29+AND+%28issecure%3A%220%22+OR+userid%3A%22c6305dc4%5C-cbba%5C-bf48%5C-97d5%5C-dcfe6f2430ef%22%29%29&facet.sort=count&hl.highlightMultiTerm=true
> Adding &hl.requireFieldMatch=true should probably solve your problem.
>
Yes, adding &hl.requireFieldMatch=true can solve my problem, but in my
solution , I have a "content" field indexing all fields' contents to
support full text search, but I also have another 2 fields "title" and
"body" which support highlight, when I do search on content, I expect
the "title" and "body" can be high-lighted. So using the
hl.requireFieldMatch=true may be not work.
Ouyang
https://sites.google.com/a/spectrum.co.jp/openinsight/
Re: Got error when range query and highlight
Posted by Ahmet Arslan <io...@yahoo.com>.
> I got an error as follows when I do a range query search
> ([1 TO *])
> on an numeric field and highlight is set on another text
> field.
Are you using hl.highlightMultiTerm=true? Pasting your search URL can give more hints.
Adding &hl.requireFieldMatch=true should probably solve your problem.