You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Igor Muntyan <ig...@gmail.com> on 2011/12/14 22:58:23 UTC
queryResultCache hit count is not being increased when
programmatically adding Lucene queries as filters in the SearchComponent
In my application I need to deal with a very large number of filter queries
that I cannot pass as http parameters - instead I add them as filters on the
ResponseBuilder:
public void process(ResponseBuilder rb)
{
List<Query> filters = rb.getFilters();
if (filters == null) {
filters = new ArrayList<Query>();
rb.setFilters(filters);
}
filters.add(userAccessQuery);
filters.add(auctionEndConditionQuery);
}
In the /admin/stats.jsp I have noticed that if the code above gets executed
then my queryResultCache hit count does not increase.
The following is the debug query:
{
"responseHeader":{
"status":0,
"QTime":31,
"params":{
"_jstate":"-NWHpuWq8R7oPBQGnJsHifjVh6blqvEe6DwUBpybB4ldLWmZEGqsSyvRk_0LX6a-U3fqO6Wd4kc",
"indent":"true",
"wt":"json",
"version":"2",
"debugQuery":"true",
"fl":"vid",
"q":"text:(Solara) OR slrDescEm:(\"Solara\") OR
vinLast8:(\"Solara\")",
"fq":"(eColorId:\"4\")"}},
"response":{"numFound":2,"start":0,"docs":[
{
"vid":18372703},
{
"vid":19071820}]
},
"debug":{
"rawquerystring":"text:(Solara) OR slrDescEm:(\"Solara\") OR
vinLast8:(\"Solara\")",
"querystring":"text:(Solara) OR slrDescEm:(\"Solara\") OR
vinLast8:(\"Solara\")",
"parsedquery":"text:solara slrDescEm:solara vinLast8:solara",
"parsedquery_toString":"text:solara slrDescEm:solara vinLast8:solara",
"explain":{
"18372703":"\n0.285029 = (MATCH) product of:\n 0.855087 = (MATCH) sum
of:\n 0.855087 = (MATCH) weight(text:solara in 4146), product of:\n
0.44694334 = queryWeight(text:solara), product of:\n 7.6527553 =
idf(docFreq=23, maxDocs=18598)\n 0.058402933 = queryNorm\n
1.9131888 = (MATCH) fieldWeight(text:solara in 4146), product of:\n
1.0 = tf(termFreq(text:solara)=1)\n 7.6527553 = idf(docFreq=23,
maxDocs=18598)\n 0.25 = fieldNorm(field=text, doc=4146)\n 0.33333334
= coord(1/3)\n",
"19071820":"\n0.285029 = (MATCH) product of:\n 0.855087 = (MATCH) sum
of:\n 0.855087 = (MATCH) weight(text:solara in 13815), product of:\n
0.44694334 = queryWeight(text:solara), product of:\n 7.6527553 =
idf(docFreq=23, maxDocs=18598)\n 0.058402933 = queryNorm\n
1.9131888 = (MATCH) fieldWeight(text:solara in 13815), product of:\n
1.0 = tf(termFreq(text:solara)=1)\n 7.6527553 = idf(docFreq=23,
maxDocs=18598)\n 0.25 = fieldNorm(field=text, doc=13815)\n
0.33333334 = coord(1/3)\n"},
"QParser":"LuceneQParser",
"filter_queries":["(eColorId:\"4\")"],
"parsed_filter_queries":["eColorId:4",
"(+(((+((+(+cgcId:4 +iter:[1 TO 6]))
/****** A VERY, VERY LONG LIST OF CONDITIONS HERE ******/
) (+(+cgcId:840 +iter:[1 TO 12])) (+(+cgcId:841 +iter:[1 TO 10]))
(+(+cgcId:843 +iter:[1 TO 12])))) +blInd:true)",
"ET:[1323899277225 TO *]"],
"timing":{
"time":31.0,
"prepare":{
"time":0.0,
"com.openlane.search.solr.filter.GenericFilter":{
"time":0.0},
"org.apache.solr.handler.component.QueryComponent":{
"time":0.0},
"org.apache.solr.handler.component.FacetComponent":{
"time":0.0},
"org.apache.solr.handler.component.MoreLikeThisComponent":{
"time":0.0},
"org.apache.solr.handler.component.HighlightComponent":{
"time":0.0},
"org.apache.solr.handler.component.StatsComponent":{
"time":0.0},
"org.apache.solr.handler.component.DebugComponent":{
"time":0.0}},
"process":{
"time":31.0,
"com.openlane.search.solr.filter.GenericFilter":{
"time":31.0},
"org.apache.solr.handler.component.QueryComponent":{
"time":0.0},
"org.apache.solr.handler.component.FacetComponent":{
"time":0.0},
"org.apache.solr.handler.component.MoreLikeThisComponent":{
"time":0.0},
"org.apache.solr.handler.component.HighlightComponent":{
"time":0.0},
"org.apache.solr.handler.component.StatsComponent":{
"time":0.0},
"org.apache.solr.handler.component.DebugComponent":{
"time":0.0}}}}}
Notice the difference between "filter_queries" and "parsed_filter_queries"
If I block filters.add(userAccessQuery) then my queryResultCache hit count
is being increased as it should. The following is the response with
debugQuery=true
in this case.
{
"responseHeader":{
"status":0,
"QTime":16,
"params":{
"indent":"true",
"wt":"json",
"version":"2",
"debugQuery":"true",
"fl":"vid",
"q":"text:(Solara) OR slrDescEm:(\"Solara\") OR
vinLast8:(\"Solara\")",
"devMode":"bypassUserAccess",
"fq":"(eColorId:\"4\")"}},
"response":{"numFound":3,"start":0,"docs":[
{
"vid":18372703},
{
"vid":19071820},
{
"vid":17192691}]
},
"debug":{
"rawquerystring":"text:(Solara) OR slrDescEm:(\"Solara\") OR
vinLast8:(\"Solara\")",
"querystring":"text:(Solara) OR slrDescEm:(\"Solara\") OR
vinLast8:(\"Solara\")",
"parsedquery":"text:solara slrDescEm:solara vinLast8:solara",
"parsedquery_toString":"text:solara slrDescEm:solara vinLast8:solara",
"explain":{
"18372703":"\n0.285029 = (MATCH) product of:\n 0.855087 = (MATCH) sum
of:\n 0.855087 = (MATCH) weight(text:solara in 4146), product of:\n
0.44694334 = queryWeight(text:solara), product of:\n 7.6527553 =
idf(docFreq=23, maxDocs=18598)\n 0.058402933 = queryNorm\n
1.9131888 = (MATCH) fieldWeight(text:solara in 4146), product of:\n
1.0 = tf(termFreq(text:solara)=1)\n 7.6527553 = idf(docFreq=23,
maxDocs=18598)\n 0.25 = fieldNorm(field=text, doc=4146)\n 0.33333334
= coord(1/3)\n",
"19071820":"\n0.285029 = (MATCH) product of:\n 0.855087 = (MATCH) sum
of:\n 0.855087 = (MATCH) weight(text:solara in 13815), product of:\n
0.44694334 = queryWeight(text:solara), product of:\n 7.6527553 =
idf(docFreq=23, maxDocs=18598)\n 0.058402933 = queryNorm\n
1.9131888 = (MATCH) fieldWeight(text:solara in 13815), product of:\n
1.0 = tf(termFreq(text:solara)=1)\n 7.6527553 = idf(docFreq=23,
maxDocs=18598)\n 0.25 = fieldNorm(field=text, doc=13815)\n
0.33333334 = coord(1/3)\n",
"17192691":"\n0.24940038 = (MATCH) product of:\n 0.74820113 = (MATCH)
sum of:\n 0.74820113 = (MATCH) weight(text:solara in 166), product of:\n
0.44694334 = queryWeight(text:solara), product of:\n 7.6527553 =
idf(docFreq=23, maxDocs=18598)\n 0.058402933 = queryNorm\n
1.6740402 = (MATCH) fieldWeight(text:solara in 166), product of:\n
1.0 = tf(termFreq(text:solara)=1)\n 7.6527553 = idf(docFreq=23,
maxDocs=18598)\n 0.21875 = fieldNorm(field=text, doc=166)\n
0.33333334 = coord(1/3)\n"},
"QParser":"LuceneQParser",
"filter_queries":["(eColorId:\"4\")"],
"parsed_filter_queries":["eColorId:4"],
"timing":{
"time":16.0,
"prepare":{
"time":16.0,
"com.openlane.search.solr.filter.GenericFilter":{
"time":0.0},
"org.apache.solr.handler.component.QueryComponent":{
"time":16.0},
"org.apache.solr.handler.component.FacetComponent":{
"time":0.0},
"org.apache.solr.handler.component.MoreLikeThisComponent":{
"time":0.0},
"org.apache.solr.handler.component.HighlightComponent":{
"time":0.0},
"org.apache.solr.handler.component.StatsComponent":{
"time":0.0},
"org.apache.solr.handler.component.DebugComponent":{
"time":0.0}},
"process":{
"time":0.0,
"com.openlane.search.solr.filter.GenericFilter":{
"time":0.0},
"org.apache.solr.handler.component.QueryComponent":{
"time":0.0},
"org.apache.solr.handler.component.FacetComponent":{
"time":0.0},
"org.apache.solr.handler.component.MoreLikeThisComponent":{
"time":0.0},
"org.apache.solr.handler.component.HighlightComponent":{
"time":0.0},
"org.apache.solr.handler.component.StatsComponent":{
"time":0.0},
"org.apache.solr.handler.component.DebugComponent":{
"time":0.0}}}}}
Is this a bug or expected behaviour?
Thanks and appreciate your help.
--
View this message in context: http://lucene.472066.n3.nabble.com/queryResultCache-hit-count-is-not-being-increased-when-programmatically-adding-Lucene-queries-as-filt-tp3586892p3586892.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: queryResultCache hit count is not being increased when
programmatically adding Lucene queries as filters in the SearchComponent
Posted by Igor Muntyan <ig...@gmail.com>.
Solr version: 3.2.0
--
View this message in context: http://lucene.472066.n3.nabble.com/queryResultCache-hit-count-is-not-being-increased-when-programmatically-adding-Lucene-queries-as-filt-tp3586892p3586904.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: queryResultCache hit count is not being increased when
programmatically adding Lucene queries as filters in the SearchComponent
Posted by Igor Muntyan <ig...@gmail.com>.
Thanks Chris,
Now it all makes sense. One of my filters is a date range filter with the
current time as a lower range, so it changes with every query. I have just
tried to comment it out and I see the queryResultCache being utilized. I
will try to round the currentTimeMillis down to the nearest minute or
something like that.
--
View this message in context: http://lucene.472066.n3.nabble.com/queryResultCache-hit-count-is-not-being-increased-when-programmatically-adding-Lucene-queries-as-filt-tp3586892p3602366.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: queryResultCache hit count is not being increased when
programmatically adding Lucene queries as filters in the SearchComponent
Posted by Chris Hostetter <ho...@fucit.org>.
: In the /admin/stats.jsp I have noticed that if the code above gets executed
: then my queryResultCache hit count does not increase.
Filters are part of the cache key for the queryResultCache (because the
cache contains the sorted paginated results *after* filters have been
applied) so if your filters are unique for every request, you're never
going to see a cache hit.
-Hoss