You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Wendy2 <we...@rcsb.org> on 2018/01/29 19:56:27 UTC
Help with Boolean search using Solr parser edismax
Hi Solr users,I am having an issue on boolean search with Solr parser
edismax. The search "OR" doesn't work. The image below shows the different
results tested on different Solr versions. There are two types of search
requester handlers, /select vs /search. The /select requester uses Lucene
default parser, while /search uses Solr edismax parser. I also listed the
search requester handler below. I am expecting the result count of 997 (844
+ 153) but I only get the correct count via the default /select request
handler on Solr v5.3.0 and 6.2.0. I I go back to use the old Lucene default
parser via /select request handler, I lose all the nice customization
ranking and sorting :-(Does anyone know some workaround/solution to fix this
type of search issue? THANKS!
<http://lucene.472066.n3.nabble.com/file/t493740/boolean-search-diff-solrs.png>
Part of the /search request handler in solrconfig.xml
file:trueexplicitedismaxpdb_id^5.0struct.title^35.0citation.title^25.0title_fields_stem^3.0.......rest_fields_stem
^0.3score desc,release_date desc,pdb_id desc7100text
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Wendy2 <we...@rcsb.org>.
Hi Erick,
Thank you very much for the clarification. I will keep it in my mind since
we are now in the process of migrating MySQL database to mongoDB.
Best Regards,
Wendy
a happy Solr user
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Wendy2 <we...@rcsb.org>.
Hi Erick,
Yes. Currently I re-index the database on a weekly basis because we only
have weekly release.
As part of the Solr weekly re-index, the batch job will delete the
/solr/core/data folder, restart Solr server, then re-index.
We use Luigi to build/control pipelines of Solr re-index batch jobs.
Thanks for al your help and support!
All the best,
Wendy
a happy Solr user :-)
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Erick Erickson <er...@gmail.com>.
From the ref guide:
"Field names should consist of alphanumeric or underscore characters
only and not start with a digit. This is not currently strictly
enforced, but other field names will not have first class support from
all components and back compatibility is not guaranteed."
You need to _completely_ blow away your index and re-index from
scratch though. By "completely blow away" I mean
1> shut down your Solrs and "rm -rf each_core/data"
or
2> create a new collection and index into that
or
3> delete your collection and re-create it.
If you just re-index your entire corpus after changing the field
names, the metadata for the old fields will be preserved. Likely you
won't notice, it's not very much data but....
Erick
On Fri, Feb 2, 2018 at 6:14 AM, Wendy2 <we...@rcsb.org> wrote:
> Good morning, Emir,
>
> Thanks for letting me know that. I used dots to add tableName. as a field
> prefix because several columns from different tables have the same names.
> In your opinion, what will be the best way to replace dots?
>
> Happy Friday!
>
> Wendy
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Wendy2 <we...@rcsb.org>.
Good morning, Emir,
Thanks for letting me know that. I used dots to add tableName. as a field
prefix because several columns from different tables have the same names.
In your opinion, what will be the best way to replace dots?
Happy Friday!
Wendy
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Erick Erickson <er...@gmail.com>.
From the reference guide:
Field names should consist of alphanumeric or underscore characters
only and not start with a digit. This is not currently strictly
enforced, but other field names will not have first class support from
all components and back compatibility is not guaranteed.
Best,
Erick
On Fri, Feb 2, 2018 at 1:07 AM, Emir Arnautović
<em...@sematext.com> wrote:
> Hi Wendy,
> A bit off-topic, but forgot to mention in previous mail: dots in field names are not recommended. Even it obviously works for you, I think I’ve seen people reporting some issues caused by dot in field names (I cannot find some reference now). So, if you plan some system upgrade in the future, you might want to get rid of field names with dots - you can safely use underscore.
>
> Regards,
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
>> On 1 Feb 2018, at 17:19, Wendy2 <we...@rcsb.org> wrote:
>>
>> And the coupon has no expiration date on it (LOL). Thank you again, Emir!
>>
>> Best Regards,
>>
>> Wendy
>>
>>
>>
>> --
>> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
Re: Help with Boolean search using Solr parser edismax
Posted by Emir Arnautović <em...@sematext.com>.
Hi Wendy,
A bit off-topic, but forgot to mention in previous mail: dots in field names are not recommended. Even it obviously works for you, I think I’ve seen people reporting some issues caused by dot in field names (I cannot find some reference now). So, if you plan some system upgrade in the future, you might want to get rid of field names with dots - you can safely use underscore.
Regards,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 1 Feb 2018, at 17:19, Wendy2 <we...@rcsb.org> wrote:
>
> And the coupon has no expiration date on it (LOL). Thank you again, Emir!
>
> Best Regards,
>
> Wendy
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Wendy2 <we...@rcsb.org>.
And the coupon has no expiration date on it (LOL). Thank you again, Emir!
Best Regards,
Wendy
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Emir Arnautović <em...@sematext.com>.
Hi Wendy,
You are welcome! I’ll put your lunch coupon in my wallet, just in case I get hungry around NJ ;)
Regards,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 1 Feb 2018, at 16:26, Wendy2 <we...@rcsb.org> wrote:
>
> Excellent!!! Thank you so much for all your help, Emir!
>
> Both worked now and I got 997 result counts back as the expected number :-)
>
> /rcsb/search?q=method:"x-ray*" "Solution NMR"&mm=1
> /rcsb/search?q=+method:"x-ray*" +"Solution NMR"&mm=1
>
> I will keep this in my mind regarding query with multiple parsers:
> /select?q=method:”x-ray*” OR _query({!edismax mm=7
> qf=‘title_field_stem^3,….’}”Solution NMR”).
>
> Thanks again and have a wonderful Thursday!
> If you ever come to NJ area, I would like to take you out for a lunch to
> thank you for all your help!
>
> Wendy
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Wendy2 <we...@rcsb.org>.
Excellent!!! Thank you so much for all your help, Emir!
Both worked now and I got 997 result counts back as the expected number :-)
/rcsb/search?q=method:"x-ray*" "Solution NMR"&mm=1
/rcsb/search?q=+method:"x-ray*" +"Solution NMR"&mm=1
I will keep this in my mind regarding query with multiple parsers:
/select?q=method:”x-ray*” OR _query({!edismax mm=7
qf=‘title_field_stem^3,….’}”Solution NMR”).
Thanks again and have a wonderful Thursday!
If you ever come to NJ area, I would like to take you out for a lunch to
thank you for all your help!
Wendy
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Emir Arnautović <em...@sematext.com>.
Hi Wendy,
Query now looks as expected but you are not getting results as expected. The reason for that is edismax’s mm parameter is what matters. You are setting it to 7 and you have two parts to match so it is always AND and you don’t have such documents. You can set it to 1 and it will be OR.
If you really need to have OR between some edismax query and some other query, you will have to use standard parser and use _query, something like:
/select?q=method:”x-ray*” OR _query({!edismax mm=7 qf=‘title_field_stem^3,….’}”Solution NMR”).
You can put it in config and use placeholders to pass values.
HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 1 Feb 2018, at 15:25, Wendy2 <we...@rcsb.org> wrote:
>
> Good morning, Emir,
>
> Here are the debug output for case 1f-a (q=method:"x-ray*" "Solution NMR"),
> 1f-b (q=+method:"x-ray*" +"Solution NMR"). both returned zero counts. It
> looks that the querystrings are the same. Thanks for following up on my
> post and your help! -- Wendy
>
>
> *=====================DebugQuery Outputs for case 1f-a,
> 1f-b=======================*
> *1f-a (/search?q=method:"x-ray*" "Solution NMR"): result count = 0*
> "debug":{
> "rawquerystring":"method:\"x-ray*\" \"Solution NMR\"",
> "querystring":"method:\"x-ray*\" \"Solution NMR\"",
> "parsedquery":"(+(PhraseQuery(method:\"x rai\")
> DisjunctionMaxQuery(((pdb_id:Solution NMR)^5.0 |
> (entity_name_com.name:\"solut nmr\")^20.0 | (citation_author.name:\"solut
> nmr\")^5.0 | (audit_author.name:\"solut nmr\")^5.0 |
> rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut nmr\")^3.0 |
> (classification:\"solut nmr\")^15.0 | (struct_keywords.text:\"solut
> nmr\")^12.0 | (entity.pdbx_description:\"solut nmr\")^10.0 |
> (pdbx_descriptor_stem:\"solut nmr\")^10.0 | (citation.title:\"solut
> nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut nmr\")^15.0 |
> (entity_src_gen_concat_stem:\"solut nmr\")^15.0 | (struct.title:\"solut
> nmr\")^35.0 | (group_id_stem:\"solut nmr\")^10.0)))~2)/no_coord",
> "parsedquery_toString":"+((method:\"x rai\" ((pdb_id:Solution NMR)^5.0 |
> (entity_name_com.name:\"solut nmr\")^20.0 | (citation_author.name:\"solut
> nmr\")^5.0 | (audit_author.name:\"solut nmr\")^5.0 |
> rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut nmr\")^3.0 |
> (classification:\"solut nmr\")^15.0 | (struct_keywords.text:\"solut
> nmr\")^12.0 | (entity.pdbx_description:\"solut nmr\")^10.0 |
> (pdbx_descriptor_stem:\"solut nmr\")^10.0 | (citation.title:\"solut
> nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut nmr\")^15.0 |
> (entity_src_gen_concat_stem:\"solut nmr\")^15.0 | (struct.title:\"solut
> nmr\")^35.0 | (group_id_stem:\"solut nmr\")^10.0))~2)",
>
>
> *1f-b (/search?q=+method:"x-ray*" +"Solution NMR") result count = 0:*
> "debug":{
> "rawquerystring":" method:\"x-ray*\" \"Solution NMR\"",
> "querystring":" method:\"x-ray*\" \"Solution NMR\"",
> "parsedquery":"(+(PhraseQuery(method:\"x rai\")
> DisjunctionMaxQuery(((pdb_id:Solution NMR)^5.0 |
> (entity_name_com.name:\"solut nmr\")^20.0 | (citation_author.name:\"solut
> nmr\")^5.0 | (audit_author.name:\"solut nmr\")^5.0 |
> rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut nmr\")^3.0 |
> (classification:\"solut nmr\")^15.0 | (struct_keywords.text:\"solut
> nmr\")^12.0 | (entity.pdbx_description:\"solut nmr\")^10.0 |
> (pdbx_descriptor_stem:\"solut nmr\")^10.0 | (citation.title:\"solut
> nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut nmr\")^15.0 |
> (entity_src_gen_concat_stem:\"solut nmr\")^15.0 | (struct.title:\"solut
> nmr\")^35.0 | (group_id_stem:\"solut nmr\")^10.0)))~2)/no_coord",
> "parsedquery_toString":"+((method:\"x rai\" ((pdb_id:Solution NMR)^5.0 |
> (entity_name_com.name:\"solut nmr\")^20.0 | (citation_author.name:\"solut
> nmr\")^5.0 | (audit_author.name:\"solut nmr\")^5.0 |
> rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut nmr\")^3.0 |
> (classification:\"solut nmr\")^15.0 | (struct_keywords.text:\"solut
> nmr\")^12.0 | (entity.pdbx_description:\"solut nmr\")^10.0 |
> (pdbx_descriptor_stem:\"solut nmr\")^10.0 | (citation.title:\"solut
> nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut nmr\")^15.0 |
> (entity_src_gen_concat_stem:\"solut nmr\")^15.0 | (struct.title:\"solut
> nmr\")^35.0 | (group_id_stem:\"solut nmr\")^10.0))~2)",
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Wendy2 <we...@rcsb.org>.
Good morning, Emir,
Here are the debug output for case 1f-a (q=method:"x-ray*" "Solution NMR"),
1f-b (q=+method:"x-ray*" +"Solution NMR"). both returned zero counts. It
looks that the querystrings are the same. Thanks for following up on my
post and your help! -- Wendy
*=====================DebugQuery Outputs for case 1f-a,
1f-b=======================*
*1f-a (/search?q=method:"x-ray*" "Solution NMR"): result count = 0*
"debug":{
"rawquerystring":"method:\"x-ray*\" \"Solution NMR\"",
"querystring":"method:\"x-ray*\" \"Solution NMR\"",
"parsedquery":"(+(PhraseQuery(method:\"x rai\")
DisjunctionMaxQuery(((pdb_id:Solution NMR)^5.0 |
(entity_name_com.name:\"solut nmr\")^20.0 | (citation_author.name:\"solut
nmr\")^5.0 | (audit_author.name:\"solut nmr\")^5.0 |
rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut nmr\")^3.0 |
(classification:\"solut nmr\")^15.0 | (struct_keywords.text:\"solut
nmr\")^12.0 | (entity.pdbx_description:\"solut nmr\")^10.0 |
(pdbx_descriptor_stem:\"solut nmr\")^10.0 | (citation.title:\"solut
nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut nmr\")^15.0 |
(entity_src_gen_concat_stem:\"solut nmr\")^15.0 | (struct.title:\"solut
nmr\")^35.0 | (group_id_stem:\"solut nmr\")^10.0)))~2)/no_coord",
"parsedquery_toString":"+((method:\"x rai\" ((pdb_id:Solution NMR)^5.0 |
(entity_name_com.name:\"solut nmr\")^20.0 | (citation_author.name:\"solut
nmr\")^5.0 | (audit_author.name:\"solut nmr\")^5.0 |
rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut nmr\")^3.0 |
(classification:\"solut nmr\")^15.0 | (struct_keywords.text:\"solut
nmr\")^12.0 | (entity.pdbx_description:\"solut nmr\")^10.0 |
(pdbx_descriptor_stem:\"solut nmr\")^10.0 | (citation.title:\"solut
nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut nmr\")^15.0 |
(entity_src_gen_concat_stem:\"solut nmr\")^15.0 | (struct.title:\"solut
nmr\")^35.0 | (group_id_stem:\"solut nmr\")^10.0))~2)",
*1f-b (/search?q=+method:"x-ray*" +"Solution NMR") result count = 0:*
"debug":{
"rawquerystring":" method:\"x-ray*\" \"Solution NMR\"",
"querystring":" method:\"x-ray*\" \"Solution NMR\"",
"parsedquery":"(+(PhraseQuery(method:\"x rai\")
DisjunctionMaxQuery(((pdb_id:Solution NMR)^5.0 |
(entity_name_com.name:\"solut nmr\")^20.0 | (citation_author.name:\"solut
nmr\")^5.0 | (audit_author.name:\"solut nmr\")^5.0 |
rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut nmr\")^3.0 |
(classification:\"solut nmr\")^15.0 | (struct_keywords.text:\"solut
nmr\")^12.0 | (entity.pdbx_description:\"solut nmr\")^10.0 |
(pdbx_descriptor_stem:\"solut nmr\")^10.0 | (citation.title:\"solut
nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut nmr\")^15.0 |
(entity_src_gen_concat_stem:\"solut nmr\")^15.0 | (struct.title:\"solut
nmr\")^35.0 | (group_id_stem:\"solut nmr\")^10.0)))~2)/no_coord",
"parsedquery_toString":"+((method:\"x rai\" ((pdb_id:Solution NMR)^5.0 |
(entity_name_com.name:\"solut nmr\")^20.0 | (citation_author.name:\"solut
nmr\")^5.0 | (audit_author.name:\"solut nmr\")^5.0 |
rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut nmr\")^3.0 |
(classification:\"solut nmr\")^15.0 | (struct_keywords.text:\"solut
nmr\")^12.0 | (entity.pdbx_description:\"solut nmr\")^10.0 |
(pdbx_descriptor_stem:\"solut nmr\")^10.0 | (citation.title:\"solut
nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut nmr\")^15.0 |
(entity_src_gen_concat_stem:\"solut nmr\")^15.0 | (struct.title:\"solut
nmr\")^35.0 | (group_id_stem:\"solut nmr\")^10.0))~2)",
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Emir Arnautović <em...@sematext.com>.
Hi Wendy,
I was thinking of query q=method:“x-ray*” “Solution NMR”
This should be equivalent to one with OR between them. If you want to put AND between those two, query would be q=+method:”x-ray*” +”Solution NMR”
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 31 Jan 2018, at 19:39, Wendy2 <we...@rcsb.org> wrote:
>
> Hi Emir,
>
> Listed below are the debugQuery outputs from query without "OR" operator. I
> really appreciate your help! --Wendy
>
> ===============DebugQuery Outputs for case 1f-a, 1f-b without "OR"
> operator=================
> *1f-a (/search?q=+method:"x-ray*" +method:"Solution NMR") result counts = 0:
> *
>
> "debug":{
> "rawquerystring":" method:\"x-ray*\" method:\"Solution NMR\"",
> "querystring":" method:\"x-ray*\" method:\"Solution NMR\"",
> "parsedquery":"(+(PhraseQuery(method:\"x rai\")
> PhraseQuery(method:\"solut nmr\"))~2)/no_coord",
> "parsedquery_toString":"+((method:\"x rai\" method:\"solut nmr\")~2)",
>
>
> *1f-b (/search?q=method:"x-ray*" method:"Solution NMR") result counts = 0: *
>
> "debug":{
> "rawquerystring":"method:\"x-ray*\" method:\"Solution NMR\"",
> "querystring":"method:\"x-ray*\" method:\"Solution NMR\"",
> "parsedquery":"(+(PhraseQuery(method:\"x rai\")
> PhraseQuery(method:\"solut nmr\"))~2)/no_coord",
> "parsedquery_toString":"+((method:\"x rai\" method:\"solut nmr\")~2)",
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Wendy2 <we...@rcsb.org>.
Hi Emir,
Listed below are the debugQuery outputs from query without "OR" operator. I
really appreciate your help! --Wendy
===============DebugQuery Outputs for case 1f-a, 1f-b without "OR"
operator=================
*1f-a (/search?q=+method:"x-ray*" +method:"Solution NMR") result counts = 0:
*
"debug":{
"rawquerystring":" method:\"x-ray*\" method:\"Solution NMR\"",
"querystring":" method:\"x-ray*\" method:\"Solution NMR\"",
"parsedquery":"(+(PhraseQuery(method:\"x rai\")
PhraseQuery(method:\"solut nmr\"))~2)/no_coord",
"parsedquery_toString":"+((method:\"x rai\" method:\"solut nmr\")~2)",
*1f-b (/search?q=method:"x-ray*" method:"Solution NMR") result counts = 0: *
"debug":{
"rawquerystring":"method:\"x-ray*\" method:\"Solution NMR\"",
"querystring":"method:\"x-ray*\" method:\"Solution NMR\"",
"parsedquery":"(+(PhraseQuery(method:\"x rai\")
PhraseQuery(method:\"solut nmr\"))~2)/no_coord",
"parsedquery_toString":"+((method:\"x rai\" method:\"solut nmr\")~2)",
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Emir Arnautović <em...@sematext.com>.
Hi Wendy,
With OR with spaces OR is interpreted as another search term. Can you try
without or - just a space between two parts. If you need and, use + before
each part.
HTH,
Emir
On Jan 31, 2018 6:24 PM, "Wendy2" <we...@rcsb.org> wrote:
Hi Emir,
Thank you so much for following up with your ticket.
Listed below are the parts of debugQuery outputs via /search request
handler. The reason I used * in the query term is that there are a couple of
methods starting with "x-ray". When I used space surrounding the "OR"
boolean search operator (refer to 1f) below, I got zero results. If I remove
the space, the result count = 19.
Thank you very much for investigating this issue. I am a happy Solr user. We
implemented Solr to our web site text search (www.rcsb.org) last year and
have improved the search results :-). Now we want to expand our text search
to support Boolean search and I am facing this issue. Thank you again for
all your help and support! --Wendy
===============DebugQuery Outputs for case 1d, 1e, 1f =================
*1d(/search?q=method:"x-ray*"):* result counts = 884
"debug":{
"rawquerystring":"method:\"x-ray*\"",
"querystring":"method:\"x-ray*\"",
"parsedquery":"(+PhraseQuery(method:\"x rai\"))/no_coord",
"parsedquery_toString":"+method:\"x rai\"",
*1e (/search?q=method:"Solution NMR"):* result counts = 153
"debug":{
"rawquerystring":"method:\"Solution NMR\"",
"querystring":"method:\"Solution NMR\"",
"parsedquery":"(+PhraseQuery(method:\"solut nmr\"))/no_coord",
"parsedquery_toString":"+method:\"solut nmr\"",
*1f (/search?q=method:"x-ray*" OR "Solution NMR"):* result counts = 0
"debug":{
"rawquerystring":"method:\"x-ray*\" OR \"Solution NMR\"",
"querystring":"method:\"x-ray*\" OR \"Solution NMR\"",
"parsedquery":"(+(PhraseQuery(method:\"x rai\")
DisjunctionMaxQuery(((pdb_id:OR)^5.0)) DisjunctionMaxQuery(((pdb_id:Solution
NMR)^5.0 | (entity_name_com.name:\"solut nmr\")^20.0 |
(citation_author.name:\"solut nmr\")^5.0 | (audit_author.name:\"solut
nmr\")^5.0 | rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut
nmr\")^3.0 | (classification:\"solut nmr\")^15.0 |
(struct_keywords.text:\"solut nmr\")^12.0 | (entity.pdbx_description:\"solut
nmr\")^10.0 | (pdbx_descriptor_stem:\"solut nmr\")^10.0 |
(citation.title:\"solut nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut
nmr\")^15.0 | (entity_src_gen_concat_stem:\"solut nmr\")^15.0 |
(struct.title:\"solut nmr\")^35.0 | (group_id_stem:\"solut
nmr\")^10.0)))~3)/no_coord",
"parsedquery_toString":"+((method:\"x rai\" ((pdb_id:OR)^5.0)
((pdb_id:Solution NMR)^5.0 | (entity_name_com.name:\"solut nmr\")^20.0 |
(citation_author.name:\"solut nmr\")^5.0 | (audit_author.name:\"solut
nmr\")^5.0 | rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut
nmr\")^3.0 | (classification:\"solut nmr\")^15.0 |
(struct_keywords.text:\"solut nmr\")^12.0 | (entity.pdbx_description:\"solut
nmr\")^10.0 | (pdbx_descriptor_stem:\"solut nmr\")^10.0 |
(citation.title:\"solut nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut
nmr\")^15.0 | (entity_src_gen_concat_stem:\"solut nmr\")^15.0 |
(struct.title:\"solut nmr\")^35.0 | (group_id_stem:\"solut
nmr\")^10.0))~3)",
"explain":{},
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Wendy2 <we...@rcsb.org>.
Hi Emir,
Thank you so much for following up with your ticket.
Listed below are the parts of debugQuery outputs via /search request
handler. The reason I used * in the query term is that there are a couple of
methods starting with "x-ray". When I used space surrounding the "OR"
boolean search operator (refer to 1f) below, I got zero results. If I remove
the space, the result count = 19.
Thank you very much for investigating this issue. I am a happy Solr user. We
implemented Solr to our web site text search (www.rcsb.org) last year and
have improved the search results :-). Now we want to expand our text search
to support Boolean search and I am facing this issue. Thank you again for
all your help and support! --Wendy
===============DebugQuery Outputs for case 1d, 1e, 1f =================
*1d(/search?q=method:"x-ray*"):* result counts = 884
"debug":{
"rawquerystring":"method:\"x-ray*\"",
"querystring":"method:\"x-ray*\"",
"parsedquery":"(+PhraseQuery(method:\"x rai\"))/no_coord",
"parsedquery_toString":"+method:\"x rai\"",
*1e (/search?q=method:"Solution NMR"):* result counts = 153
"debug":{
"rawquerystring":"method:\"Solution NMR\"",
"querystring":"method:\"Solution NMR\"",
"parsedquery":"(+PhraseQuery(method:\"solut nmr\"))/no_coord",
"parsedquery_toString":"+method:\"solut nmr\"",
*1f (/search?q=method:"x-ray*" OR "Solution NMR"):* result counts = 0
"debug":{
"rawquerystring":"method:\"x-ray*\" OR \"Solution NMR\"",
"querystring":"method:\"x-ray*\" OR \"Solution NMR\"",
"parsedquery":"(+(PhraseQuery(method:\"x rai\")
DisjunctionMaxQuery(((pdb_id:OR)^5.0)) DisjunctionMaxQuery(((pdb_id:Solution
NMR)^5.0 | (entity_name_com.name:\"solut nmr\")^20.0 |
(citation_author.name:\"solut nmr\")^5.0 | (audit_author.name:\"solut
nmr\")^5.0 | rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut
nmr\")^3.0 | (classification:\"solut nmr\")^15.0 |
(struct_keywords.text:\"solut nmr\")^12.0 | (entity.pdbx_description:\"solut
nmr\")^10.0 | (pdbx_descriptor_stem:\"solut nmr\")^10.0 |
(citation.title:\"solut nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut
nmr\")^15.0 | (entity_src_gen_concat_stem:\"solut nmr\")^15.0 |
(struct.title:\"solut nmr\")^35.0 | (group_id_stem:\"solut
nmr\")^10.0)))~3)/no_coord",
"parsedquery_toString":"+((method:\"x rai\" ((pdb_id:OR)^5.0)
((pdb_id:Solution NMR)^5.0 | (entity_name_com.name:\"solut nmr\")^20.0 |
(citation_author.name:\"solut nmr\")^5.0 | (audit_author.name:\"solut
nmr\")^5.0 | rest_fields_stem:\"solut nmr\" | (title_fields_stem:\"solut
nmr\")^3.0 | (classification:\"solut nmr\")^15.0 |
(struct_keywords.text:\"solut nmr\")^12.0 | (entity.pdbx_description:\"solut
nmr\")^10.0 | (pdbx_descriptor_stem:\"solut nmr\")^10.0 |
(citation.title:\"solut nmr\")^25.0 | (struct_keywords.pdbx_keywords:\"solut
nmr\")^15.0 | (entity_src_gen_concat_stem:\"solut nmr\")^15.0 |
(struct.title:\"solut nmr\")^35.0 | (group_id_stem:\"solut
nmr\")^10.0))~3)",
"explain":{},
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Emir Arnautović <em...@sematext.com>.
Hi Wendy,
I see several issues, but not sure if any of them is the reason why you are not getting what you expect:
* there are no spaces around OR and that results in query being parsed sometimes with OR, e.g. (pdb_id:OR\”Solution)^5
* wildcard in quotes - it is not handled as you expected - the whole phrase is analysed and wildcard is eliminated and resulting query is method:”x rai”
* In 1d you search for “x-reays*” - that will search all fields, not just method field - maybe that is why you get 844 results.
Can you provide debug query for all three 1d, 1e and 1f (with spaces around OR). Please paste results as text and not as picture, and do not update original post since some of us are using mails and we are not getting updates.
Thanks,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 30 Jan 2018, at 21:33, Wendy2 <we...@rcsb.org> wrote:
>
> Hi Emlr,
>
> Thank you for reading my post and for your reply. I updated my post with
> debug info and a better view of the definition of /search request handler.
>
> Any suggestion on what I should try?
>
> Thanks,
>
> Wendy
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Wendy2 <we...@rcsb.org>.
Hi Emlr,
Thank you for reading my post and for your reply. I updated my post with
debug info and a better view of the definition of /search request handler.
Any suggestion on what I should try?
Thanks,
Wendy
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Wendy2 <we...@rcsb.org>.
Hi Emir,
Thank you so much for your response. I updated my post with an image which
display the configuration of the /search request handler. Any suggestions?
Thanks,
Wendy
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Help with Boolean search using Solr parser edismax
Posted by Emir Arnautović <em...@sematext.com>.
Hi Wendy,
It is most likely that you need to list fields that can appear in query using uf. The best way to see what is going on is to use debugQuery and you can see more details how your query is parsed.
HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 29 Jan 2018, at 20:56, Wendy2 <we...@rcsb.org> wrote:
>
> Hi Solr users,I am having an issue on boolean search with Solr parser
> edismax. The search "OR" doesn't work. The image below shows the different
> results tested on different Solr versions. There are two types of search
> requester handlers, /select vs /search. The /select requester uses Lucene
> default parser, while /search uses Solr edismax parser. I also listed the
> search requester handler below. I am expecting the result count of 997 (844
> + 153) but I only get the correct count via the default /select request
> handler on Solr v5.3.0 and 6.2.0. I I go back to use the old Lucene default
> parser via /select request handler, I lose all the nice customization
> ranking and sorting :-(Does anyone know some workaround/solution to fix this
> type of search issue? THANKS!
> <http://lucene.472066.n3.nabble.com/file/t493740/boolean-search-diff-solrs.png>
> Part of the /search request handler in solrconfig.xml
> file:trueexplicitedismaxpdb_id^5.0struct.title^35.0citation.title^25.0title_fields_stem^3.0.......rest_fields_stem
> ^0.3score desc,release_date desc,pdb_id desc7100text
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html