You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Kura <ku...@tangentlabs.co.uk> on 2010/09/08 11:53:14 UTC

Solr searching "harri" finds "harry"

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hey,

I've got a Solr server up and running over about 10 million rows, I have
a text column that I use for my main search and a couple of int/str
fields used for faceting.

Whenever I search for the term "harri" as q=title:harri it will match
the term "harry", is there any way to stop this other than wrapping the
query in quotes?

Also, with the spellcheck enabled, trying both content based
spellchecking and jarowinkler if I deliberately misspell "harry" to
"harru" both spellchecks return a single spelling of "harri" rather than
harry. I've tried using <str name="comparatorClass">freq</str> and <str
name="comparatorClass">score</str> but none of these push harry to the
top, even though I have far more indexes with harry in than harri.

Any ideas?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAkyHXQoACgkQLOut9Un89NlhbwCfVXmOYM7mEc3XwLk3Z4AL4SP5
/0MAn1g5IgfoEpR1os/QAQ3IDSr2jRGk
=7AQm
-----END PGP SIGNATURE-----

Re: Solr searching "harri" finds "harry"

Posted by "Grijesh.singh" <pi...@gmail.com>.
yes reindexing is necessary for protwords,synanym update

-----
Grijesh
-- 
View this message in context: http://lucene.472066.n3.nabble.com/Solr-searching-harri-finds-harry-tp1438486p1438802.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr searching "harri" finds "harry"

Posted by Kura <ku...@tangentlabs.co.uk>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Yes to restart, no to re-index. Was hoping that wouldn't be necessary.
I'll do that now.

On 08/09/10 11:48, Grijesh.singh wrote:
> 
> have u restart the solr after adding words in protwords and reindex the data?
> 
> -----
> Grijesh
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAkyHbEEACgkQLOut9Un89NmAaACfdl5P/GOikHvBHu0A9/6ma30q
jXYAoIAbN8tAnMc4ecqwJ4Q8r/Un3Cio
=vmU8
-----END PGP SIGNATURE-----

Re: Solr searching "harri" finds "harry"

Posted by "Grijesh.singh" <pi...@gmail.com>.
have u restart the solr after adding words in protwords and reindex the data?

-----
Grijesh
-- 
View this message in context: http://lucene.472066.n3.nabble.com/Solr-searching-harri-finds-harry-tp1438486p1438735.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr searching "harri" finds "harry"

Posted by Kura <ku...@tangentlabs.co.uk>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I have "harry" as a protected word in protword.txt

Here is the xml definition for my text column

<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
    <analyzer type="index">
        <tokenizer class="solr.WhitespaceTokenizerFactory" />
        <filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt" />
        <filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="0" splitOnCaseChange="1" />
        <filter class="solr.LowerCaseFilterFactory" />
        <filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt" />
        <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
    </analyzer>
    <analyzer type="query">
        <tokenizer class="solr.WhitespaceTokenizerFactory" />
        <filter class="solr.SynonymFilterFactory"
synonyms="synonyms.txt" ignoreCase="true" expand="true" />
        <filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt" />
        <filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="0"
catenateNumbers="0" catenateAll="0" splitOnCaseChange="1" />
        <filter class="solr.LowerCaseFilterFactory" />
        <filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt" />
        <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
    </analyzer>
</fieldType>

On 08/09/10 11:29, Grijesh.singh wrote:
> 
> U have not provided much detail about analysis of that field,but I am sure
> that problem because of stemming
> u can see by analysis page or by debugQuery=on parameter.
> 
> To prevent stemming u have to put words in protword.txt on which u do not
> need any stemming
> 
> -----
> Grijesh
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAkyHaHIACgkQLOut9Un89NmR6wCgjOS+znMEqUQKn3ACzWudAaa4
faMAn2d0LX76ZBmiL+j/EtmVpvIpHiub
=5ymy
-----END PGP SIGNATURE-----

Re: Solr searching "harri" finds "harry"

Posted by "Grijesh.singh" <pi...@gmail.com>.
U have not provided much detail about analysis of that field,but I am sure
that problem because of stemming
u can see by analysis page or by debugQuery=on parameter.

To prevent stemming u have to put words in protword.txt on which u do not
need any stemming

-----
Grijesh
-- 
View this message in context: http://lucene.472066.n3.nabble.com/Solr-searching-harri-finds-harry-tp1438486p1438637.html
Sent from the Solr - User mailing list archive at Nabble.com.