Re: Query in quotes cannot find results

[Shawn Heisey <ap...@elyograg.org>]

On 6/30/2020 12:07 PM, Permakoff, Vadim wrote:
> Regarding removing the stopwords, I agree, there are many cases when you don't want to remove the stopwords, but there is one very compelling case when you want them to be removed.
> 
> Imagine, you have one document with the following text:
> 1. "to expand the methods for mailing cancellation"
> And another document with the text:
> 2. "to expand methods for mailing cancellation"
> 
> The user query is (without quotes): q=expand the methods for mailing cancellation
> I don't want to bring all the documents with condition q.op=OR, it will find too many unrelated documents, so I want to search with q.op=AND. Unfortunately, the document 2 will not be found as it has no stop word "the" in it.
> What should I do now?

Do these users want imprecise matches to only show up when there is a 
well-known stopword involved, or do they also want imprecise matches to 
show up with ANY word missing, added, or moved?  If I were betting on 
it, I'd say they want the latter, not the former.  Erick already gave 
you the solution to that -- phrase slop.

In modern times, the only valid reason I can think of to implement a 
stopword filter is for situations where you want it to be impossible to 
search for certain words -- some might want expletives in this category, 
for example.

Tuning a Solr config for good results is an exercise in tradeoffs.  The 
core tradeoff in most situations is the standard "precision vs. recall" 
discussion.  A change that increases precision will almost always reduce 
recall, and vice versa.  I know from experience that you'll get more 
complaints about reducing recall than you will about reducing precision. 
  Implementing a hard-coded phrase slop value of 1 will reduce precision 
by an amount that's hard to determine, and GREATLY increase recall. 
Chances are good that most users will appreciate the change.  If you 
make the phrase slop setting configurable by the user, that's even better.

Thanks,
Shawn