You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Smiley, David W." <ds...@mitre.org> on 2011/11/07 20:32:16 UTC
Add "hint" to Solr QParser that it is a filter query
I am implementing a custom Solr FieldType for Solr trunk. One method I override is:
Query getFieldQuery(QParser parser, SchemaField field, String externalVal)
My Query implementation can be vastly more efficient if the scores aren't needed vs. when they are needed. My code constructing this is basically:
Filter f = makeFilter(...);//efficient
ValueSource vs = makeValueSource(...);//potentially not needed and uses memory
return new FilteredQuery( new FunctionQuery(vs), f );
I would love to know when it is okay to return a ConstantScoreQuery wrapping my Filter so that I needn't bother with my ValueSource. In my opinion, FieldType should have a getFieldFilter() method similar to getFieldQuery(). Perhaps a hint of some kind could be added to the QParser handed in -- a boolean flag or some special local-param to indicate the constant score to use. Solr could perform this when the QParser is constructed for a Filter Query.
Thoughts? Am I missing something?
~ David
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Add "hint" to Solr QParser that it is a filter query
Posted by "Smiley, David W." <ds...@mitre.org>.
On Nov 7, 2011, at 3:41 PM, Chris Hostetter wrote:
>
> : I would love to know when it is okay to return a ConstantScoreQuery
> : wrapping my Filter so that I needn't bother with my ValueSource. In my
> : opinion, FieldType should have a getFieldFilter() method similar to
> : getFieldQuery(). Perhaps a hint of some kind could be added to the
> : QParser handed in -- a boolean flag or some special local-param to
> : indicate the constant score to use. Solr could perform this when the
> : QParser is constructed for a Filter Query.
>
> my gut raction is that this seems like it could be dangerous in the
> general case -- it could lead to subtle bugs where a FieldType might
> inadvertantly produce a query that matches different documents depending
> on context (doesn't mean we shouldn't do it, just a risk i worry about)
>
> if we do go this route, adding it as a hint on the QParser seems like a
> cleaner appraoch then a completely new method on FieldType (particularly
> because then generic QParsers might be able to leverage this info as well
> ... consider fq={!geofilt ... } vs q={!geofilt ... }).
The QParser hint is definitely a practical solution that I can use right now as you suggest below. I'd like to see QueryComponent introduce this hint when it creates the Query from the fq param. I'll create a JIRA issue.
My proposal of a FieldType.getFilter() method is not a mutually exclusive suggestion with the QParser hint. FieldType already has a getFieldQuery(), getRangeQuery() and getValueSource() methods... a getFilter() method seems oddly missing to me. With this in place, and a QParser hint in place as well, the getFieldQuery() method in FieldType could check for the hint and return a new ConstantScoreQuery(getFilter(...)). If the hint is false then another method could be delegated to for FieldType subclasses that need custom behavior. Perhaps this addition would facilitate Solr's support for LUCENE-1536 "if a filter can support random access API, we should use it" I am not sure honestly.
> As a first pass i would suggest that your FieldType could do this kind of
> logic as is right now by reqiring a special param (local or otherwise),
> something like...
>
> fq={!field needScore=false f=yourSpecialField}foo
>
> ...where your field type has...
>
> public Query getFieldQuery(QParser parser, SchemaField field, String externalVal)
> Filter f = makeFilter(...);
> if (parser.getLocalParams().getBool("needScore", true)) {
> ValueSource vs = makeValueSource(...);
> uses memory
> return new FilteredQuery( new FunctionQuery(vs), f );
> }
> return new ConstantScoreQuery(f);
> }
>
> ...in general this type of explicit approach (based on a param) is
> apealing because it would help mitigate the risk i mentioned about queries
> matching diff docs being (accidently?) produced depending on context --
> much easier to spot/understand if the query changes if-and-only-if a param
> changes.
>
>
>
> -Hoss
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Add "hint" to Solr QParser that it is a filter query
Posted by Chris Hostetter <ho...@fucit.org>.
: I would love to know when it is okay to return a ConstantScoreQuery
: wrapping my Filter so that I needn't bother with my ValueSource. In my
: opinion, FieldType should have a getFieldFilter() method similar to
: getFieldQuery(). Perhaps a hint of some kind could be added to the
: QParser handed in -- a boolean flag or some special local-param to
: indicate the constant score to use. Solr could perform this when the
: QParser is constructed for a Filter Query.
my gut raction is that this seems like it could be dangerous in the
general case -- it could lead to subtle bugs where a FieldType might
inadvertantly produce a query that matches different documents depending
on context (doesn't mean we shouldn't do it, just a risk i worry about)
if we do go this route, adding it as a hint on the QParser seems like a
cleaner appraoch then a completely new method on FieldType (particularly
because then generic QParsers might be able to leverage this info as well
... consider fq={!geofilt ... } vs q={!geofilt ... }).
As a first pass i would suggest that your FieldType could do this kind of
logic as is right now by reqiring a special param (local or otherwise),
something like...
fq={!field needScore=false f=yourSpecialField}foo
...where your field type has...
public Query getFieldQuery(QParser parser, SchemaField field, String externalVal)
Filter f = makeFilter(...);
if (parser.getLocalParams().getBool("needScore", true)) {
ValueSource vs = makeValueSource(...);
uses memory
return new FilteredQuery( new FunctionQuery(vs), f );
}
return new ConstantScoreQuery(f);
}
...in general this type of explicit approach (based on a param) is
apealing because it would help mitigate the risk i mentioned about queries
matching diff docs being (accidently?) produced depending on context --
much easier to spot/understand if the query changes if-and-only-if a param
changes.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org