You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Aaron Gibbons <ag...@patriotsoftware.com> on 2018/08/15 15:23:36 UTC
Solr changing the search when given many qf fields?
I found a tipping point where the search being built changes with the
number of qf fields being passed in.
Example search: "foo bar"
solr 7.2.1
select?q.op=AND&defType=edismax&q=foo bar
Debugging the query you can see it results in:
"parsedquery_toString":"+(+(text:foo) +(text:bar))"
Adding more qf values you get:
"text name_text"
"parsedquery_toString":"+(+(name_text:foo | text:foo) +(name_text:bar |
text:bar))"
"text name_text city_text"
"parsedquery_toString":"+(+(city_text:foo | name_text:foo | text:foo)
+(city_text:bar | name_text:bar | text:bar))"
The search continues to build this way until I get to a certain amount of
qf values.
Large number of qf values:
"34 values.."
"parsedquery_toString":"+(+((+comments_text:foo +comments_text:bar) |
(+zip_text:foo +zip_text:bar) | (+city_text:foo +city_text:bar) |
(+street_address_text:foo +street_address_text:bar) |
(+street_address_two_text:foo +street_address_two_text:bar) |
(+state_text:foo +state_text:bar)..."
Now the search is requiring both foo and bar to be in each qf field in the
search, not foo to be in any qf field and bar to be in any qf field. I had
to cut the number of qf values down to 15 to get it back to the correct
search.
Why is the search changing? Is there any way around this or a better way we
should be doing the search?
I realize we could copy all of the fields to the default text field. However,
most of the fields are searchable individually as well as keyword
searchable so specifying the fields vs using the default text field makes
sense in that respect.
Thank you,
Aaron
Re: Solr changing the search when given many qf fields?
Posted by Aaron Gibbons <ag...@patriotsoftware.com>.
Thank you Emir! This was certainly the case.
One field was processed with EdgeNGramTokenizerFactory and should not have
been.
Fixed.
On Thu, Aug 16, 2018 at 4:46 AM, Emir Arnautović <
emir.arnautovic@sematext.com> wrote:
> Hi Aaron,
> It is probably not about number of fields but related to different
> analysis of different fields. As long as all your fields analyzers produce
> the same tokens you should get “term centric” query. Once any of your
> analyzers produce different token, it’ll become “field centric”. It is
> likely that one of your fields (tipping point) is string and produces “foo
> bar” token.
> Here is blog post explaining this part of edismax in a context of multi
> term synonyms: https://opensourceconnections.com/blog/2018/02/20/edismax-
> and-multiterm-synonyms-oddities/ <https://opensourceconnections.com/
> blog/2018/02/20/edismax-and-multiterm-synonyms-oddities/>
>
> HTH,
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
> > On 15 Aug 2018, at 17:23, Aaron Gibbons <ag...@patriotsoftware.com>
> wrote:
> >
> > I found a tipping point where the search being built changes with the
> > number of qf fields being passed in.
> >
> > Example search: "foo bar"
> > solr 7.2.1
> > select?q.op=AND&defType=edismax&q=foo bar
> >
> > Debugging the query you can see it results in:
> > "parsedquery_toString":"+(+(text:foo) +(text:bar))"
> >
> > Adding more qf values you get:
> > "text name_text"
> > "parsedquery_toString":"+(+(name_text:foo | text:foo) +(name_text:bar |
> > text:bar))"
> > "text name_text city_text"
> > "parsedquery_toString":"+(+(city_text:foo | name_text:foo | text:foo)
> > +(city_text:bar | name_text:bar | text:bar))"
> >
> > The search continues to build this way until I get to a certain amount of
> > qf values.
> >
> > Large number of qf values:
> > "34 values.."
> > "parsedquery_toString":"+(+((+comments_text:foo +comments_text:bar) |
> > (+zip_text:foo +zip_text:bar) | (+city_text:foo +city_text:bar) |
> > (+street_address_text:foo +street_address_text:bar) |
> > (+street_address_two_text:foo +street_address_two_text:bar) |
> > (+state_text:foo +state_text:bar)..."
> > Now the search is requiring both foo and bar to be in each qf field in
> the
> > search, not foo to be in any qf field and bar to be in any qf field. I
> had
> > to cut the number of qf values down to 15 to get it back to the correct
> > search.
> >
> > Why is the search changing? Is there any way around this or a better way
> we
> > should be doing the search?
> > I realize we could copy all of the fields to the default text field.
> However,
> > most of the fields are searchable individually as well as keyword
> > searchable so specifying the fields vs using the default text field makes
> > sense in that respect.
> >
> > Thank you,
> > Aaron
>
>
Re: Solr changing the search when given many qf fields?
Posted by Emir Arnautović <em...@sematext.com>.
Hi Aaron,
It is probably not about number of fields but related to different analysis of different fields. As long as all your fields analyzers produce the same tokens you should get “term centric” query. Once any of your analyzers produce different token, it’ll become “field centric”. It is likely that one of your fields (tipping point) is string and produces “foo bar” token.
Here is blog post explaining this part of edismax in a context of multi term synonyms: https://opensourceconnections.com/blog/2018/02/20/edismax-and-multiterm-synonyms-oddities/ <https://opensourceconnections.com/blog/2018/02/20/edismax-and-multiterm-synonyms-oddities/>
HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 15 Aug 2018, at 17:23, Aaron Gibbons <ag...@patriotsoftware.com> wrote:
>
> I found a tipping point where the search being built changes with the
> number of qf fields being passed in.
>
> Example search: "foo bar"
> solr 7.2.1
> select?q.op=AND&defType=edismax&q=foo bar
>
> Debugging the query you can see it results in:
> "parsedquery_toString":"+(+(text:foo) +(text:bar))"
>
> Adding more qf values you get:
> "text name_text"
> "parsedquery_toString":"+(+(name_text:foo | text:foo) +(name_text:bar |
> text:bar))"
> "text name_text city_text"
> "parsedquery_toString":"+(+(city_text:foo | name_text:foo | text:foo)
> +(city_text:bar | name_text:bar | text:bar))"
>
> The search continues to build this way until I get to a certain amount of
> qf values.
>
> Large number of qf values:
> "34 values.."
> "parsedquery_toString":"+(+((+comments_text:foo +comments_text:bar) |
> (+zip_text:foo +zip_text:bar) | (+city_text:foo +city_text:bar) |
> (+street_address_text:foo +street_address_text:bar) |
> (+street_address_two_text:foo +street_address_two_text:bar) |
> (+state_text:foo +state_text:bar)..."
> Now the search is requiring both foo and bar to be in each qf field in the
> search, not foo to be in any qf field and bar to be in any qf field. I had
> to cut the number of qf values down to 15 to get it back to the correct
> search.
>
> Why is the search changing? Is there any way around this or a better way we
> should be doing the search?
> I realize we could copy all of the fields to the default text field. However,
> most of the fields are searchable individually as well as keyword
> searchable so specifying the fields vs using the default text field makes
> sense in that respect.
>
> Thank you,
> Aaron