query parser question

[Joe Calderon <ca...@gmail.com>]

i have field called text_stem that has a kstemmer on it, im having
trouble matching wildcard searches on a word that got stemmed

for example i index the word "america's", which according to
analysis.jsp after stemming gets indexed as "america"

when matching i do a query like myfield:(ame*) which matches the
indexed term, this all works fine until the query becomes
myfield:(america's*) at which point it doesnt match, however if i
remove the wildcard like myfield:(america's) the it works again

its almost like the term doesnt get stemmed when using a wildcard

im using 1.4 nightly, is this the correct behaviour, is there
something i should do differently?

in the mean time ive added "americas" as protected word in the
kstemmer but im afraid of more edge cases that will come up

--joe

Re: query parser question

[Yonik Seeley <yo...@lucidimagination.com>]

On Thu, Sep 10, 2009 at 1:28 PM, Joe Calderon <ca...@gmail.com> wrote:
> i have field called text_stem that has a kstemmer on it, im having
> trouble matching wildcard searches on a word that got stemmed
>
> for example i index the word "america's", which according to
> analysis.jsp after stemming gets indexed as "america"
>
> when matching i do a query like myfield:(ame*) which matches the
> indexed term, this all works fine until the query becomes
> myfield:(america's*) at which point it doesnt match, however if i
> remove the wildcard like myfield:(america's) the it works again
>
> its almost like the term doesnt get stemmed when using a wildcard

Correct - it's not stemmed.  If it were stemmed, there would be
multiple cases where that wouldn't work either.

For example, with the porter stemmer, "any"->"ani" and "anywhere"->"anywher"

So if you had a document with "anywhere", a prefix query of "any*"
wouldn't work if you stemmed it, and would match other things like
"animal".

-Yonik
http://www.lucidimagination.com