Matching exact words

[ahammad <ah...@gmail.com>]

Hello,

I have a case where if I search for the word "windows", I get results
containing both "windows" and "window" (and probably other things like
"windowing" etc.). Is there a way to find exact matches only?

The field in which I am searching is a text field, which as I understand
causes this behaviour. I cannot use a string field because it is very
restricted, but what else can be done? I understand there are other types of
text fields that are more strict than the standard field.

Ideally I would like to keep my index the way it is, with the ability to
force exact matches. For example, if I can search "windows -window" or
something like that, that would be great. Or if I can wrap my query in a set
of quotes to tell it to match exactly. I've seen that done before but I
cannot get it to work.

As a reference, here is my query:

q={!boost b=$db v=$qq
defType=$sh}&qq=windows&db=recip(ms(NOW,lastModifiedLong),3.16e-11,1,1)&sh=dismax

To be quite frank, I am not very familiar with this syntax. I am just using
whatever my old coworker left behind. 

Any tips on how to find exact matches or improve the above query will be
greatly appreciated.

Thanks
-- 
View this message in context: http://lucene.472066.n3.nabble.com/Matching-exact-words-tp1353350p1353350.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Matching exact words

[Erick Erickson <er...@gmail.com>]

See below:

On Thu, Aug 26, 2010 at 10:24 AM, ahammad <ah...@gmail.com> wrote:

>
> Hello Erick,
>
> Thanks for the reply. I am a little confused by this whole stemming thing.
> What exactly does it refer to?
>

In your schema file, for the "text" field type, you'll see a line like:
<filter class="solr.SnowballPorterFilterFactory" language="English"
protected="protwords.txt"/>

Which inserts the a stemmer in your filter chain. Stemmers
algorithmically reduce words to their root, e.g. running,
runs, etc all reduce to run. The reduced term is all that's
put in your index. And when you search, assuming it goes
through the same analysis chain, your query will look for
run too. The analysis admin page is your friend here for
understanding how all this goes together. See:
<your solr server url here>/solr/admin/analysis.jsp


>
> Basically, I already have a field which is essentially a collection of many
> other fields (done using copyField). This field is a text field. So what
> you're saying is to have a duplicate of this field with different
> properties
> such that it does not stem?
>
> This is pretty much what I was suggesting, but whether it's appropriate
for your situation is up to you. Making a duplicate field may be
prohibitive,
can't tell without knowing more about your problem space.




> When querying, I assume that I will have to explicitly specify which field
> to search against...is this correct?
>
> Yep, or use the dismax request handler, it lets you do this automagically.
The  dismax request handler is probably the thing you should look at first,
it lets you configure searches to look at multiple fields with different
boosts...

Best
Erick


> I'm a little rusty on the solr stuff to be honest so please bear with me.
>
> Thanks
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Matching-exact-words-tp1353350p1357027.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Matching exact words

[ahammad <ah...@gmail.com>]

Hello Erick,

Thanks for the reply. I am a little confused by this whole stemming thing.
What exactly does it refer to?

Basically, I already have a field which is essentially a collection of many
other fields (done using copyField). This field is a text field. So what
you're saying is to have a duplicate of this field with different properties
such that it does not stem?

When querying, I assume that I will have to explicitly specify which field
to search against...is this correct?

I'm a little rusty on the solr stuff to be honest so please bear with me.

Thanks
-- 
View this message in context: http://lucene.472066.n3.nabble.com/Matching-exact-words-tp1353350p1357027.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Matching exact words

[Erick Erickson <er...@gmail.com>]

You'll have to change your index I'm afraid. The problem is
that all the index sees is the stemmed version (assuming
you're stemming at index time). There's no information in
the index about what the original version was, so it's impossible
to back this out.

One solution is to use copyfield to make a copy of the
input that does NOT stem, and search against (or boost)
that field when you care about stemmed/unstemmed.

And a minor clarification. The "types" you refer to aren't
really a SOLR entity. They are just a convenient collection
of tokenizers and stemmers that are provided in the schema
file. You can freely create your own types by simply mixing and
matching various varieties of these (you probably already know
this, but the phrasing of your question caused me to wonder).

See:
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters

Best
Erick

On Thu, Aug 26, 2010 at 7:24 AM, ahammad <ah...@gmail.com> wrote:

>
> Hello,
>
> I have a case where if I search for the word "windows", I get results
> containing both "windows" and "window" (and probably other things like
> "windowing" etc.). Is there a way to find exact matches only?
>
> The field in which I am searching is a text field, which as I understand
> causes this behaviour. I cannot use a string field because it is very
> restricted, but what else can be done? I understand there are other types
> of
> text fields that are more strict than the standard field.
>
> Ideally I would like to keep my index the way it is, with the ability to
> force exact matches. For example, if I can search "windows -window" or
> something like that, that would be great. Or if I can wrap my query in a
> set
> of quotes to tell it to match exactly. I've seen that done before but I
> cannot get it to work.
>
> As a reference, here is my query:
>
> q={!boost b=$db v=$qq
>
> defType=$sh}&qq=windows&db=recip(ms(NOW,lastModifiedLong),3.16e-11,1,1)&sh=dismax
>
> To be quite frank, I am not very familiar with this syntax. I am just using
> whatever my old coworker left behind.
>
> Any tips on how to find exact matches or improve the above query will be
> greatly appreciated.
>
> Thanks
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Matching-exact-words-tp1353350p1353350.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>