You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Greg Pendlebury <gr...@gmail.com> on 2016/06/01 01:47:51 UTC
Re: After Solr 5.5, mm parameter doesn't work properly
I don't think it is 8812. q.op was completely ignored by edismax prior to
5.5, so it is not mm that changed.
If you do the same 5.4 query with q.op=OR I suspect it will not change the
debug query at all.
On 30 May 2016 at 21:07, Jan Høydahl <ja...@cominvent.com> wrote:
> Hi,
>
> This may be related to SOLR-8812, but still different. Please file a JIRA
> issue for this.
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> > 29. mai 2016 kl. 18.20 skrev Issei Nishigata <du...@gmail.com>:
> >
> > Hi,
> >
> > “mm" parameter does not work properly, when I set "q.op=AND” after Solr
> 5.5.
> > In Solr 5.4, mm parameter works expectedly with the following setting.
> >
> > ---
> > [schema]
> > <fieldType name="bi_gram" class="solr.TextField">
> > <analyzer>
> > <tokenizer class="solr.NGramTokenizerFactory" minGramSize="2"
> maxGramSize="2"/>
> > </analyzer>
> > </fieldType>
> >
> >
> > [request]
> >
> http://localhost:8983/solr/collection1/select?defType=edismax&q.op=AND&mm=2&q=solar
> > —
> >
> > After Solr 5.5, the result will not be the same as Solr 5.4.
> > Has the setting of mm parameter specs, or description of file setting
> changed?
> >
> >
> > [Solr 5.4]
> > <response>
> > ...
> > <lst name="params">
> > <str name="mm">2</str>
> > <str name="q">solar</str>
> > <str name="defType">edismax</str>
> > <str name="q.op">AND</str>
> > </lst>
> > ...
> > <result name="response" numFound="1" start="0">
> > <doc>
> > <str name="id">0</str>
> > <arr name="content">
> > <str>solr</str>
> > </arr>
> > </doc>
> > </result>
> > <lst name="debug">
> > <str name="rawquerystring">solar</str>
> > <str name="querystring">solar</str>
> > <str name="parsedquery">
> > (+DisjunctionMaxQuery((((text:so text:ol text:la
> text:ar)~2))))/no_coord
> > </str>
> > <str name="parsedquery_toString">+(((text:so text:ol text:la
> text:ar)~2))</str>
> > ...
> > </response>
> >
> >
> >
> > [Solr 6.0.1]
> >
> > <response>
> > ...
> > <lst name="params">
> > <str name="mm">2</str>
> > <str name="q">solar</str>
> > <str name="defType">edismax</str>
> > <str name="q.op">AND</str>
> > </lst>
> > ...
> > <result name="response" numFound="0" start="0"/>
> > <lst name="debug">
> > <str name="rawquerystring">solar</str>
> > <str name="querystring">solar</str>
> > <str name="parsedquery">
> > (+DisjunctionMaxQuery(((+text:so +text:ol +text:la
> +text:ar))))/no_coord
> > </str>
> > <str name="parsedquery_toString">+((+text:so +text:ol +text:la
> +text:ar))</str>
> > ...
> >
> >
> > As shown above, parsedquery also differs from Solr 5.4 and Solr
> 6.0.1(after Solr 5.5).
> >
> >
> > —
> > Thanks
> > Issei Nishigata
>
>
Re: After Solr 5.5, mm parameter doesn't work properly
Posted by Greg Pendlebury <gr...@gmail.com>.
I think the confusion stems from the legacy implementation partially
conflating q.op with mm for users, when they are very different things.
q.op tells Solr how to insert boolean operators before they are converted
into occurs flags, and then downstream, mm applies on _only_ the SHOULD
occurs flags, not MUST or NOT flags.
So if the user is setting mm=2, they are asking for a minimum of 2 of the
SHOULD clauses to be found, not 2 of ALL clauses. mm has absolutely nothing
to do with q.op other than (because of the implementation) q.op is used to
derive a default value when it is not explicitly set.
The legacy implementation has situations where it was not possible to
generate the search you wanted because of the conflation, hence why
SOLR-2649 was so popular. I fully acknowledge that there are cases where
the change is disrupting users that (for whatever reason) are/were not
necessarily aware of what the parameters they are using actually do, or
users that were very aware, but forced to rely on a non-intuitive settings
to work around the behaviour eDismax had. SOLR-8812 (although not relevant
to the OP) goes part way towards helping the former users, but the latter
will want to adjust their parameters to be explicit now instead of
leveraging a workaround.
I haven't yet seen a use case where the final solution we put in for
SOLR-2649 does not work, but I have seen lots of user parameters used that
Solr handles perfectly... just in a way that the user did not expect. I
suspect this is mainly because the topic and the implementation are fairly
technically dense (from q.op, then to boolean to occurs conversion, then
finally to mm) and difficult to explain and document accurately for an end
user.
I am writing this in rush sorry, to go collect a child from school.
Ta,
Greg
On 2 June 2016 at 19:08, Jan Høydahl <ja...@cominvent.com> wrote:
> [Aside] Your quote style is confusing, leaving my lines unquoted and your
> new lines quoted?? [/Aside]
>
> > So in relation to the OP's sample queries I was pointing out that
> 'q.op=OR
> > + mm=2' and 'q,op=AND + mm=2' are treated as identical queries by Solr
> 5.4,
> > but 5.5+ will manipulate the occurs flags differently before it applies
> mm
> > afterwards... because that is what q.op does.
>
> If a user explicitly says mm=2, then the users intent is that he should
> neither have pure OR (no clauses required) nor pure AND (all clauses
> required),
> but exactly two clauses required.
>
> So I think we need to go back to a solution where q.op technically
> stays as OR for custom mm. How that would affect queries with explicit
> operators
> I don’t know...
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> > 2. jun. 2016 kl. 05.12 skrev Greg Pendlebury <greg.pendlebury@gmail.com
> >:
> >
> > I would describe that subtly differently, and I think it is where the
> > difference lies:
> >
> > "Then from 4.x it did not care about q.op if mm was set explicitly"
> >>> I agree. q.op was not actually used in the query, but rather as a way
> of
> > inferred the default mm value. eDismax still ignored whatever q.op was
> set
> > and built your query operators (ie. the occurs flags) using q.op=OR.
> >
> > "And from 5.5 it seems as q.op does something even if mm is set..."
> >>> Yes, although I think it is the words 'even if' drawing too strong a
> > relationship between the two parameters. q.op has a function of its own,
> > and that now functions as it 'should' (opinionated, I know) in the query
> > construction, and continues to influence the default value of mm if it
> has
> > not been explicitly set. SOLR-8812 further evolves that influence by
> trying
> > to improve backwards compatibility for users who were not explicitly
> > setting mm, and only ever changed 'q.op' despite it being a step removed
> > from the actual parameter they were trying to manipulate.
> >
> > So in relation to the OP's sample queries I was pointing out that
> 'q.op=OR
> > + mm=2' and 'q,op=AND + mm=2' are treated as identical queries by Solr
> 5.4,
> > but 5.5+ will manipulate the occurs flags differently before it applies
> mm
> > afterwards... because that is what q.op does.
> >
> >
> > On 2 June 2016 at 07:13, Jan Høydahl <ja...@cominvent.com> wrote:
> >
> >> Edismax used to default to mm=100% and not care about q.op at all
> >>
> >> Then from 4.x it did not care about q.op if mm was set explicitly,
> >> but if mm was not set, then q.op=OR —> mm=0%, q.op=AND —> mm=100%
> >>
> >> And from 5.5 it seems as q.op does something even if mm is set...
> >>
> >> --
> >> Jan Høydahl, search solution architect
> >> Cominvent AS - www.cominvent.com
> >>
> >>> 1. jun. 2016 kl. 23.05 skrev Greg Pendlebury <
> greg.pendlebury@gmail.com
> >>> :
> >>>
> >>> But isn't that the default value? In this case the OP is setting mm
> >>> explicitly to 2.
> >>>
> >>> Will have to look at those code links more thoroughly at work this
> >> morning.
> >>> Apologies if I am wrong.
> >>>
> >>> Ta,
> >>> Greg
> >>>
> >>> On Wednesday, 1 June 2016, Jan Høydahl <ja...@cominvent.com> wrote:
> >>>
> >>>>> 1. jun. 2016 kl. 03.47 skrev Greg Pendlebury <
> >> greg.pendlebury@gmail.com
> >>>> <javascript:;>>:
> >>>>
> >>>>> I don't think it is 8812. q.op was completely ignored by edismax
> prior
> >> to
> >>>>> 5.5, so it is not mm that changed.
> >>>>
> >>>> That is not the case. Prior to 5.5, mm would be automatically set to
> >> 100%
> >>>> if q.op==AND
> >>>> See https://issues.apache.org/jira/browse/SOLR-1889 and
> >>>> https://svn.apache.org/viewvc?view=revision&revision=950710
> >>>>
> >>>> Jan
> >>
> >>
>
>
Re: After Solr 5.5, mm parameter doesn't work properly
Posted by Jan Høydahl <ja...@cominvent.com>.
[Aside] Your quote style is confusing, leaving my lines unquoted and your new lines quoted?? [/Aside]
> So in relation to the OP's sample queries I was pointing out that 'q.op=OR
> + mm=2' and 'q,op=AND + mm=2' are treated as identical queries by Solr 5.4,
> but 5.5+ will manipulate the occurs flags differently before it applies mm
> afterwards... because that is what q.op does.
If a user explicitly says mm=2, then the users intent is that he should
neither have pure OR (no clauses required) nor pure AND (all clauses required),
but exactly two clauses required.
So I think we need to go back to a solution where q.op technically
stays as OR for custom mm. How that would affect queries with explicit operators
I don’t know...
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
> 2. jun. 2016 kl. 05.12 skrev Greg Pendlebury <gr...@gmail.com>:
>
> I would describe that subtly differently, and I think it is where the
> difference lies:
>
> "Then from 4.x it did not care about q.op if mm was set explicitly"
>>> I agree. q.op was not actually used in the query, but rather as a way of
> inferred the default mm value. eDismax still ignored whatever q.op was set
> and built your query operators (ie. the occurs flags) using q.op=OR.
>
> "And from 5.5 it seems as q.op does something even if mm is set..."
>>> Yes, although I think it is the words 'even if' drawing too strong a
> relationship between the two parameters. q.op has a function of its own,
> and that now functions as it 'should' (opinionated, I know) in the query
> construction, and continues to influence the default value of mm if it has
> not been explicitly set. SOLR-8812 further evolves that influence by trying
> to improve backwards compatibility for users who were not explicitly
> setting mm, and only ever changed 'q.op' despite it being a step removed
> from the actual parameter they were trying to manipulate.
>
> So in relation to the OP's sample queries I was pointing out that 'q.op=OR
> + mm=2' and 'q,op=AND + mm=2' are treated as identical queries by Solr 5.4,
> but 5.5+ will manipulate the occurs flags differently before it applies mm
> afterwards... because that is what q.op does.
>
>
> On 2 June 2016 at 07:13, Jan Høydahl <ja...@cominvent.com> wrote:
>
>> Edismax used to default to mm=100% and not care about q.op at all
>>
>> Then from 4.x it did not care about q.op if mm was set explicitly,
>> but if mm was not set, then q.op=OR —> mm=0%, q.op=AND —> mm=100%
>>
>> And from 5.5 it seems as q.op does something even if mm is set...
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>>
>>> 1. jun. 2016 kl. 23.05 skrev Greg Pendlebury <greg.pendlebury@gmail.com
>>> :
>>>
>>> But isn't that the default value? In this case the OP is setting mm
>>> explicitly to 2.
>>>
>>> Will have to look at those code links more thoroughly at work this
>> morning.
>>> Apologies if I am wrong.
>>>
>>> Ta,
>>> Greg
>>>
>>> On Wednesday, 1 June 2016, Jan Høydahl <ja...@cominvent.com> wrote:
>>>
>>>>> 1. jun. 2016 kl. 03.47 skrev Greg Pendlebury <
>> greg.pendlebury@gmail.com
>>>> <javascript:;>>:
>>>>
>>>>> I don't think it is 8812. q.op was completely ignored by edismax prior
>> to
>>>>> 5.5, so it is not mm that changed.
>>>>
>>>> That is not the case. Prior to 5.5, mm would be automatically set to
>> 100%
>>>> if q.op==AND
>>>> See https://issues.apache.org/jira/browse/SOLR-1889 and
>>>> https://svn.apache.org/viewvc?view=revision&revision=950710
>>>>
>>>> Jan
>>
>>
Re: After Solr 5.5, mm parameter doesn't work properly
Posted by Greg Pendlebury <gr...@gmail.com>.
I would describe that subtly differently, and I think it is where the
difference lies:
"Then from 4.x it did not care about q.op if mm was set explicitly"
>> I agree. q.op was not actually used in the query, but rather as a way of
inferred the default mm value. eDismax still ignored whatever q.op was set
and built your query operators (ie. the occurs flags) using q.op=OR.
"And from 5.5 it seems as q.op does something even if mm is set..."
>> Yes, although I think it is the words 'even if' drawing too strong a
relationship between the two parameters. q.op has a function of its own,
and that now functions as it 'should' (opinionated, I know) in the query
construction, and continues to influence the default value of mm if it has
not been explicitly set. SOLR-8812 further evolves that influence by trying
to improve backwards compatibility for users who were not explicitly
setting mm, and only ever changed 'q.op' despite it being a step removed
from the actual parameter they were trying to manipulate.
So in relation to the OP's sample queries I was pointing out that 'q.op=OR
+ mm=2' and 'q,op=AND + mm=2' are treated as identical queries by Solr 5.4,
but 5.5+ will manipulate the occurs flags differently before it applies mm
afterwards... because that is what q.op does.
On 2 June 2016 at 07:13, Jan Høydahl <ja...@cominvent.com> wrote:
> Edismax used to default to mm=100% and not care about q.op at all
>
> Then from 4.x it did not care about q.op if mm was set explicitly,
> but if mm was not set, then q.op=OR —> mm=0%, q.op=AND —> mm=100%
>
> And from 5.5 it seems as q.op does something even if mm is set...
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> > 1. jun. 2016 kl. 23.05 skrev Greg Pendlebury <greg.pendlebury@gmail.com
> >:
> >
> > But isn't that the default value? In this case the OP is setting mm
> > explicitly to 2.
> >
> > Will have to look at those code links more thoroughly at work this
> morning.
> > Apologies if I am wrong.
> >
> > Ta,
> > Greg
> >
> > On Wednesday, 1 June 2016, Jan Høydahl <ja...@cominvent.com> wrote:
> >
> >>> 1. jun. 2016 kl. 03.47 skrev Greg Pendlebury <
> greg.pendlebury@gmail.com
> >> <javascript:;>>:
> >>
> >>> I don't think it is 8812. q.op was completely ignored by edismax prior
> to
> >>> 5.5, so it is not mm that changed.
> >>
> >> That is not the case. Prior to 5.5, mm would be automatically set to
> 100%
> >> if q.op==AND
> >> See https://issues.apache.org/jira/browse/SOLR-1889 and
> >> https://svn.apache.org/viewvc?view=revision&revision=950710
> >>
> >> Jan
>
>
Re: After Solr 5.5, mm parameter doesn't work properly
Posted by Jan Høydahl <ja...@cominvent.com>.
Edismax used to default to mm=100% and not care about q.op at all
Then from 4.x it did not care about q.op if mm was set explicitly,
but if mm was not set, then q.op=OR —> mm=0%, q.op=AND —> mm=100%
And from 5.5 it seems as q.op does something even if mm is set...
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
> 1. jun. 2016 kl. 23.05 skrev Greg Pendlebury <gr...@gmail.com>:
>
> But isn't that the default value? In this case the OP is setting mm
> explicitly to 2.
>
> Will have to look at those code links more thoroughly at work this morning.
> Apologies if I am wrong.
>
> Ta,
> Greg
>
> On Wednesday, 1 June 2016, Jan Høydahl <ja...@cominvent.com> wrote:
>
>>> 1. jun. 2016 kl. 03.47 skrev Greg Pendlebury <greg.pendlebury@gmail.com
>> <javascript:;>>:
>>
>>> I don't think it is 8812. q.op was completely ignored by edismax prior to
>>> 5.5, so it is not mm that changed.
>>
>> That is not the case. Prior to 5.5, mm would be automatically set to 100%
>> if q.op==AND
>> See https://issues.apache.org/jira/browse/SOLR-1889 and
>> https://svn.apache.org/viewvc?view=revision&revision=950710
>>
>> Jan
Re: After Solr 5.5, mm parameter doesn't work properly
Posted by Greg Pendlebury <gr...@gmail.com>.
But isn't that the default value? In this case the OP is setting mm
explicitly to 2.
Will have to look at those code links more thoroughly at work this morning.
Apologies if I am wrong.
Ta,
Greg
On Wednesday, 1 June 2016, Jan Høydahl <ja...@cominvent.com> wrote:
> > 1. jun. 2016 kl. 03.47 skrev Greg Pendlebury <greg.pendlebury@gmail.com
> <javascript:;>>:
>
> > I don't think it is 8812. q.op was completely ignored by edismax prior to
> > 5.5, so it is not mm that changed.
>
> That is not the case. Prior to 5.5, mm would be automatically set to 100%
> if q.op==AND
> See https://issues.apache.org/jira/browse/SOLR-1889 and
> https://svn.apache.org/viewvc?view=revision&revision=950710
>
> Jan
Re: After Solr 5.5, mm parameter doesn't work properly
Posted by Jan Høydahl <ja...@cominvent.com>.
> 1. jun. 2016 kl. 03.47 skrev Greg Pendlebury <gr...@gmail.com>:
> I don't think it is 8812. q.op was completely ignored by edismax prior to
> 5.5, so it is not mm that changed.
That is not the case. Prior to 5.5, mm would be automatically set to 100% if q.op==AND
See https://issues.apache.org/jira/browse/SOLR-1889 and https://svn.apache.org/viewvc?view=revision&revision=950710
Jan