You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Tommy Chheng <to...@gmail.com> on 2010/06/18 01:13:42 UTC
dismax and AND as the default operator
I'm using the dismax request handler and want to set the default
operator to AND.
Using the standard handler, i could just use the q.op or defaultOperator
in the schema, but this doesn't work using the dismax request handler.
For example, if I call "solr/select/?q=fuel+cell", I want solr to handle
it as a "solr/select/?q=fuel+AND+cell"
--
@tommychheng
Programmer and UC Irvine Graduate Student
Find a great grad school based on research interests: http://gradschoolnow.com
Re: dismax and AND as the default operator
Posted by Jan Høydahl / Cominvent <ja...@cominvent.com>.
Standard DisMax does not fully support explicit AND/OR.
You can prove that by trying to say q=fuel+OR+cell and see that the score stays the same (given mm=100%)
It appears that DisMax does SOME intelligent handling of AND/OR/NOT, because it adds the "+" on the AND and a "-" on the NOT. But adding a "+" is redundant and does not change anything as long as mm=100%. The NOT actually seems to work, but the OR does not have any effect due to the "+" on the top-level ().
If you need boolean syntax support in DisMax, try the defType=edismax with patch SOLR-1553 or alternatively on branch_3x
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Training in Europe - www.solrtraining.com
On 18. juni 2010, at 02.44, Erik Hatcher wrote:
> Hmmm, maybe I'm wrong and it does support AND. Looking at the code I don't see why it wouldn't, actually. Though I believe I've seen it documented that it isn't supported (or at least not advertised to support). Ok, from the dismax wiki page it says: "This query handler supports an extremely simplified subset of the Lucene QueryParser syntax. Quotes can be used to group phrases, and +/- can be used to denote mandatory and optional clauses". Only special single characters are escaped. So AND/OR must work. Learn something new every day!
>
> Erik
>
>
>
> On Jun 17, 2010, at 8:28 PM, Tommy Chheng wrote:
>
>> Thanks, Erik. that does work. I misunderstood the documentation, i thought "clause" meant "field" rather than the terms in the query.
>>
>> If dismax doesn't support the operator AND, why would the query "solr/select?q=fuel+cell" and "solr/select?q=fuel+AND+cell" get parsed differently(it adds the + for the AND query) and have different result count?
>>
>> @tommychheng
>> Programmer and UC Irvine Graduate Student
>> Find a great grad school based on research interests: http://gradschoolnow.com
>>
>>
>> On 6/17/10 5:17 PM, Erik Hatcher wrote:
>>> dismax does not support the operator AND. It uses +/- only.
>>>
>>> set mm=100% (not 1), as Hoss said, and try your query again.
>>>
>>> Erik
>>>
>>> On Jun 17, 2010, at 8:08 PM, Tommy Chheng wrote:
>>>
>>>> I don't think setting the mm helps.
>>>> I have mm to 1 which means the query terms should be in at least one field. Both query strings satisfy this condition.
>>>>
>>>> The query "solr/select?q=fuel+cell" is parsed as
>>>>
>>>> "querystring":"fuel cell",
>>>> "parsedquery":"+((DisjunctionMaxQuery((text:fuel | organization_name_ws_lc:fuel^5.0)) DisjunctionMaxQuery((text:cell | organization_name_ws_lc:cell^5.0)))~1) ()",
>>>> "parsedquery_toString":"+(((text:fuel | organization_name_ws_lc:fuel^5.0) (text:cell | organization_name_ws_lc:cell^5.0))~1) ()",
>>>>
>>>> returns ~900 results
>>>>
>>>> The query "solr/select?q=fuel+AND+cell" is parsed as
>>>>
>>>> "querystring":"fuel AND cell",
>>>> "parsedquery":"+(+DisjunctionMaxQuery((text:fuel | organization_name_ws_lc:fuel^5.0)) +DisjunctionMaxQuery((text:cell | organization_name_ws_lc:cell^5.0))) ()",
>>>> "parsedquery_toString":"+(+(text:fuel | organization_name_ws_lc:fuel^5.0) +(text:cell | organization_name_ws_lc:cell^5.0)) ()",
>>>> returns ~80 results
>>>>
>>>> (this is the behavior i want for query "fuel cell" because it adds the extra +). I want to do this without adding the AND for every query.
>>>>
>>>>
>>>>
>>>> @tommychheng
>>>> Programmer and UC Irvine Graduate Student
>>>> Find a great grad school based on research interests: http://gradschoolnow.com
>>>>
>>>>
>>>> On 6/17/10 4:19 PM, Chris Hostetter wrote:
>>>>> : I'm using the dismax request handler and want to set the default operator to
>>>>> : AND.
>>>>> : Using the standard handler, i could just use the q.op or defaultOperator in
>>>>> : the schema, but this doesn't work using the dismax request handler.
>>>>> :
>>>>> : For example, if I call "solr/select/?q=fuel+cell", I want solr to handle it as
>>>>> : a "solr/select/?q=fuel+AND+cell"
>>>>>
>>>>> Please consult the dismax docs...
>>>>> http://wiki.apache.org/solr/DisMaxRequestHandler#mm_.28Minimum_.27Should.27_Match.29
>>>>>
>>>>> dismax uses the "mm" param to decide how clauses that don't have an
>>>>> explicit operator will be dealt with -- the default is to require 100% of
>>>>> the terms, so if you aren't seeing that behavior then you have a
>>>>> solrconfig.xml that that sets the default mm value to something else.
>>>>>
>>>>> Starting with Solr 4.0 (and mybe 3.1 if it's backported) the default mm
>>>>> will be based on the value of q.op (see SOLR-1889 for more details)
>>>>>
>>>>>
>>>>> -Hoss
>>>>>
>>>
>
Re: dismax and AND as the default operator
Posted by Erik Hatcher <er...@gmail.com>.
Hmmm, maybe I'm wrong and it does support AND. Looking at the code I
don't see why it wouldn't, actually. Though I believe I've seen it
documented that it isn't supported (or at least not advertised to
support). Ok, from the dismax wiki page it says: "This query handler
supports an extremely simplified subset of the Lucene QueryParser
syntax. Quotes can be used to group phrases, and +/- can be used to
denote mandatory and optional clauses". Only special single
characters are escaped. So AND/OR must work. Learn something new
every day!
Erik
On Jun 17, 2010, at 8:28 PM, Tommy Chheng wrote:
> Thanks, Erik. that does work. I misunderstood the documentation, i
> thought "clause" meant "field" rather than the terms in the query.
>
> If dismax doesn't support the operator AND, why would the query
> "solr/select?q=fuel+cell" and "solr/select?q=fuel+AND+cell" get
> parsed differently(it adds the + for the AND query) and have
> different result count?
>
> @tommychheng
> Programmer and UC Irvine Graduate Student
> Find a great grad school based on research interests: http://gradschoolnow.com
>
>
> On 6/17/10 5:17 PM, Erik Hatcher wrote:
>> dismax does not support the operator AND. It uses +/- only.
>>
>> set mm=100% (not 1), as Hoss said, and try your query again.
>>
>> Erik
>>
>> On Jun 17, 2010, at 8:08 PM, Tommy Chheng wrote:
>>
>>> I don't think setting the mm helps.
>>> I have mm to 1 which means the query terms should be in at least
>>> one field. Both query strings satisfy this condition.
>>>
>>> The query "solr/select?q=fuel+cell" is parsed as
>>>
>>> "querystring":"fuel cell",
>>> "parsedquery":"+((DisjunctionMaxQuery((text:fuel |
>>> organization_name_ws_lc:fuel^5.0)) DisjunctionMaxQuery((text:cell
>>> | organization_name_ws_lc:cell^5.0)))~1) ()",
>>> "parsedquery_toString":"+(((text:fuel |
>>> organization_name_ws_lc:fuel^5.0) (text:cell |
>>> organization_name_ws_lc:cell^5.0))~1) ()",
>>>
>>> returns ~900 results
>>>
>>> The query "solr/select?q=fuel+AND+cell" is parsed as
>>>
>>> "querystring":"fuel AND cell",
>>> "parsedquery":"+(+DisjunctionMaxQuery((text:fuel |
>>> organization_name_ws_lc:fuel^5.0)) +DisjunctionMaxQuery((text:cell
>>> | organization_name_ws_lc:cell^5.0))) ()",
>>> "parsedquery_toString":"+(+(text:fuel |
>>> organization_name_ws_lc:fuel^5.0) +(text:cell |
>>> organization_name_ws_lc:cell^5.0)) ()",
>>> returns ~80 results
>>>
>>> (this is the behavior i want for query "fuel cell" because it adds
>>> the extra +). I want to do this without adding the AND for every
>>> query.
>>>
>>>
>>>
>>> @tommychheng
>>> Programmer and UC Irvine Graduate Student
>>> Find a great grad school based on research interests: http://gradschoolnow.com
>>>
>>>
>>> On 6/17/10 4:19 PM, Chris Hostetter wrote:
>>>> : I'm using the dismax request handler and want to set the
>>>> default operator to
>>>> : AND.
>>>> : Using the standard handler, i could just use the q.op or
>>>> defaultOperator in
>>>> : the schema, but this doesn't work using the dismax request
>>>> handler.
>>>> :
>>>> : For example, if I call "solr/select/?q=fuel+cell", I want solr
>>>> to handle it as
>>>> : a "solr/select/?q=fuel+AND+cell"
>>>>
>>>> Please consult the dismax docs...
>>>> http://wiki.apache.org/solr/DisMaxRequestHandler#mm_.28Minimum_.27Should.27_Match.29
>>>>
>>>> dismax uses the "mm" param to decide how clauses that don't have an
>>>> explicit operator will be dealt with -- the default is to require
>>>> 100% of
>>>> the terms, so if you aren't seeing that behavior then you have a
>>>> solrconfig.xml that that sets the default mm value to something
>>>> else.
>>>>
>>>> Starting with Solr 4.0 (and mybe 3.1 if it's backported) the
>>>> default mm
>>>> will be based on the value of q.op (see SOLR-1889 for more details)
>>>>
>>>>
>>>> -Hoss
>>>>
>>
Re: dismax and AND as the default operator
Posted by Tommy Chheng <to...@gmail.com>.
Thanks, Erik. that does work. I misunderstood the documentation, i
thought "clause" meant "field" rather than the terms in the query.
If dismax doesn't support the operator AND, why would the query
"solr/select?q=fuel+cell" and "solr/select?q=fuel+AND+cell" get parsed
differently(it adds the + for the AND query) and have different result
count?
@tommychheng
Programmer and UC Irvine Graduate Student
Find a great grad school based on research interests: http://gradschoolnow.com
On 6/17/10 5:17 PM, Erik Hatcher wrote:
> dismax does not support the operator AND. It uses +/- only.
>
> set mm=100% (not 1), as Hoss said, and try your query again.
>
> Erik
>
> On Jun 17, 2010, at 8:08 PM, Tommy Chheng wrote:
>
>> I don't think setting the mm helps.
>> I have mm to 1 which means the query terms should be in at least one
>> field. Both query strings satisfy this condition.
>>
>> The query "solr/select?q=fuel+cell" is parsed as
>>
>> "querystring":"fuel cell",
>> "parsedquery":"+((DisjunctionMaxQuery((text:fuel |
>> organization_name_ws_lc:fuel^5.0)) DisjunctionMaxQuery((text:cell |
>> organization_name_ws_lc:cell^5.0)))~1) ()",
>> "parsedquery_toString":"+(((text:fuel |
>> organization_name_ws_lc:fuel^5.0) (text:cell |
>> organization_name_ws_lc:cell^5.0))~1) ()",
>>
>> returns ~900 results
>>
>> The query "solr/select?q=fuel+AND+cell" is parsed as
>>
>> "querystring":"fuel AND cell",
>> "parsedquery":"+(+DisjunctionMaxQuery((text:fuel |
>> organization_name_ws_lc:fuel^5.0)) +DisjunctionMaxQuery((text:cell |
>> organization_name_ws_lc:cell^5.0))) ()",
>> "parsedquery_toString":"+(+(text:fuel |
>> organization_name_ws_lc:fuel^5.0) +(text:cell |
>> organization_name_ws_lc:cell^5.0)) ()",
>> returns ~80 results
>>
>> (this is the behavior i want for query "fuel cell" because it adds
>> the extra +). I want to do this without adding the AND for every query.
>>
>>
>>
>> @tommychheng
>> Programmer and UC Irvine Graduate Student
>> Find a great grad school based on research interests:
>> http://gradschoolnow.com
>>
>>
>> On 6/17/10 4:19 PM, Chris Hostetter wrote:
>>> : I'm using the dismax request handler and want to set the default
>>> operator to
>>> : AND.
>>> : Using the standard handler, i could just use the q.op or
>>> defaultOperator in
>>> : the schema, but this doesn't work using the dismax request handler.
>>> :
>>> : For example, if I call "solr/select/?q=fuel+cell", I want solr to
>>> handle it as
>>> : a "solr/select/?q=fuel+AND+cell"
>>>
>>> Please consult the dismax docs...
>>> http://wiki.apache.org/solr/DisMaxRequestHandler#mm_.28Minimum_.27Should.27_Match.29
>>>
>>>
>>> dismax uses the "mm" param to decide how clauses that don't have an
>>> explicit operator will be dealt with -- the default is to require
>>> 100% of
>>> the terms, so if you aren't seeing that behavior then you have a
>>> solrconfig.xml that that sets the default mm value to something else.
>>>
>>> Starting with Solr 4.0 (and mybe 3.1 if it's backported) the default mm
>>> will be based on the value of q.op (see SOLR-1889 for more details)
>>>
>>>
>>> -Hoss
>>>
>
Re: dismax and AND as the default operator
Posted by Erik Hatcher <er...@gmail.com>.
dismax does not support the operator AND. It uses +/- only.
set mm=100% (not 1), as Hoss said, and try your query again.
Erik
On Jun 17, 2010, at 8:08 PM, Tommy Chheng wrote:
> I don't think setting the mm helps.
> I have mm to 1 which means the query terms should be in at least one
> field. Both query strings satisfy this condition.
>
> The query "solr/select?q=fuel+cell" is parsed as
>
> "querystring":"fuel cell",
> "parsedquery":"+((DisjunctionMaxQuery((text:fuel |
> organization_name_ws_lc:fuel^5.0)) DisjunctionMaxQuery((text:cell |
> organization_name_ws_lc:cell^5.0)))~1) ()",
> "parsedquery_toString":"+(((text:fuel |
> organization_name_ws_lc:fuel^5.0) (text:cell |
> organization_name_ws_lc:cell^5.0))~1) ()",
>
> returns ~900 results
>
> The query "solr/select?q=fuel+AND+cell" is parsed as
>
> "querystring":"fuel AND cell",
> "parsedquery":"+(+DisjunctionMaxQuery((text:fuel |
> organization_name_ws_lc:fuel^5.0)) +DisjunctionMaxQuery((text:cell |
> organization_name_ws_lc:cell^5.0))) ()",
> "parsedquery_toString":"+(+(text:fuel |
> organization_name_ws_lc:fuel^5.0) +(text:cell |
> organization_name_ws_lc:cell^5.0)) ()",
> returns ~80 results
>
> (this is the behavior i want for query "fuel cell" because it adds
> the extra +). I want to do this without adding the AND for every
> query.
>
>
>
> @tommychheng
> Programmer and UC Irvine Graduate Student
> Find a great grad school based on research interests: http://gradschoolnow.com
>
>
> On 6/17/10 4:19 PM, Chris Hostetter wrote:
>> : I'm using the dismax request handler and want to set the default
>> operator to
>> : AND.
>> : Using the standard handler, i could just use the q.op or
>> defaultOperator in
>> : the schema, but this doesn't work using the dismax request handler.
>> :
>> : For example, if I call "solr/select/?q=fuel+cell", I want solr to
>> handle it as
>> : a "solr/select/?q=fuel+AND+cell"
>>
>> Please consult the dismax docs...
>> http://wiki.apache.org/solr/DisMaxRequestHandler#mm_.28Minimum_.27Should.27_Match.29
>>
>> dismax uses the "mm" param to decide how clauses that don't have an
>> explicit operator will be dealt with -- the default is to require
>> 100% of
>> the terms, so if you aren't seeing that behavior then you have a
>> solrconfig.xml that that sets the default mm value to something else.
>>
>> Starting with Solr 4.0 (and mybe 3.1 if it's backported) the
>> default mm
>> will be based on the value of q.op (see SOLR-1889 for more details)
>>
>>
>> -Hoss
>>
Re: dismax and AND as the default operator
Posted by Tommy Chheng <to...@gmail.com>.
I don't think setting the mm helps.
I have mm to 1 which means the query terms should be in at least one
field. Both query strings satisfy this condition.
The query "solr/select?q=fuel+cell" is parsed as
"querystring":"fuel cell",
"parsedquery":"+((DisjunctionMaxQuery((text:fuel | organization_name_ws_lc:fuel^5.0)) DisjunctionMaxQuery((text:cell | organization_name_ws_lc:cell^5.0)))~1) ()",
"parsedquery_toString":"+(((text:fuel | organization_name_ws_lc:fuel^5.0) (text:cell | organization_name_ws_lc:cell^5.0))~1) ()",
returns ~900 results
The query "solr/select?q=fuel+AND+cell" is parsed as
"querystring":"fuel AND cell",
"parsedquery":"+(+DisjunctionMaxQuery((text:fuel | organization_name_ws_lc:fuel^5.0)) +DisjunctionMaxQuery((text:cell | organization_name_ws_lc:cell^5.0))) ()",
"parsedquery_toString":"+(+(text:fuel | organization_name_ws_lc:fuel^5.0) +(text:cell | organization_name_ws_lc:cell^5.0)) ()",
returns ~80 results
(this is the behavior i want for query "fuel cell" because it adds the extra +). I want to do this without adding the AND for every query.
@tommychheng
Programmer and UC Irvine Graduate Student
Find a great grad school based on research interests: http://gradschoolnow.com
On 6/17/10 4:19 PM, Chris Hostetter wrote:
> : I'm using the dismax request handler and want to set the default operator to
> : AND.
> : Using the standard handler, i could just use the q.op or defaultOperator in
> : the schema, but this doesn't work using the dismax request handler.
> :
> : For example, if I call "solr/select/?q=fuel+cell", I want solr to handle it as
> : a "solr/select/?q=fuel+AND+cell"
>
> Please consult the dismax docs...
> http://wiki.apache.org/solr/DisMaxRequestHandler#mm_.28Minimum_.27Should.27_Match.29
>
> dismax uses the "mm" param to decide how clauses that don't have an
> explicit operator will be dealt with -- the default is to require 100% of
> the terms, so if you aren't seeing that behavior then you have a
> solrconfig.xml that that sets the default mm value to something else.
>
> Starting with Solr 4.0 (and mybe 3.1 if it's backported) the default mm
> will be based on the value of q.op (see SOLR-1889 for more details)
>
>
> -Hoss
>
Re: dismax and AND as the default operator
Posted by Chris Hostetter <ho...@fucit.org>.
: I'm using the dismax request handler and want to set the default operator to
: AND.
: Using the standard handler, i could just use the q.op or defaultOperator in
: the schema, but this doesn't work using the dismax request handler.
:
: For example, if I call "solr/select/?q=fuel+cell", I want solr to handle it as
: a "solr/select/?q=fuel+AND+cell"
Please consult the dismax docs...
http://wiki.apache.org/solr/DisMaxRequestHandler#mm_.28Minimum_.27Should.27_Match.29
dismax uses the "mm" param to decide how clauses that don't have an
explicit operator will be dealt with -- the default is to require 100% of
the terms, so if you aren't seeing that behavior then you have a
solrconfig.xml that that sets the default mm value to something else.
Starting with Solr 4.0 (and mybe 3.1 if it's backported) the default mm
will be based on the value of q.op (see SOLR-1889 for more details)
-Hoss