You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by mahendra mahendra <ma...@yahoo.com> on 2009/02/02 08:27:13 UTC
Exact match search problem
Hi,
I have indexed my data as "custom123, customer, custom" for the "UserName" field.
I need to search the records for exact match, when I am trying to search with UserName:"customer" I am finding the records where UserName is custom123 and custom.
As per my understanding solr splits the AlphaNumeric words into sub words
custom123 => "custom","123"
As per above the above logic when I search for UserName:"customer", it shouldn't display the custom123 and custom.
Could you please tell me why it is behaving like that or how I can search for exact match.
I am using the following declaration for text field in schema file
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<!-- in this example, we will only use synonyms at query time
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="false"/>
-->
<!-- Case insensitive stop word removal.
enablePositionIncrements=true ensures that a 'gap' is left to
allow for accurate phrase queries.
-->
<filter class="solr.StopFilterFactory"
ignoreCase="true"
words="englishstopwords.txt"
enablePositionIncrements="true"
/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory" protected="englishprotwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="englishsynonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="englishstopwords.txt"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory" protected="englishprotwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
Thanks in advance!!
- Mahendra
Re: Exact match search problem
Posted by Stephen Weiss <sw...@stylesight.com>.
Try using fieldtype "string" instead of "text" for the UserName
field. Then it will not be tokenized so it should only give exact
matches.
--
Steve
On Feb 2, 2009, at 2:27 AM, mahendra mahendra wrote:
> Hi,
>
> I have indexed my data as "custom123, customer, custom" for the
> "UserName" field.
> I need to search the records for exact match, when I am trying to
> search with UserName:"customer" I am finding the records where
> UserName is custom123 and custom.
>
> As per my understanding solr splits the AlphaNumeric words into sub
> words
> custom123 => "custom","123"
>
> As per above the above logic when I search for UserName:"customer",
> it shouldn't display the custom123 and custom.
>
> Could you please tell me why it is behaving like that or how I can
> search for exact match.
>
> I am using the following declaration for text field in schema file
>
> <fieldType name="text" class="solr.TextField"
> positionIncrementGap="100">
> <analyzer type="index">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <!-- in this example, we will only use synonyms at query time
> <filter class="solr.SynonymFilterFactory"
> synonyms="synonyms.txt" ignoreCase="true" expand="false"/>
> -->
> <!-- Case insensitive stop word removal.
> enablePositionIncrements=true ensures that a 'gap' is
> left to
> allow for accurate phrase queries.
> -->
> <filter class="solr.StopFilterFactory"
> ignoreCase="true"
> words="englishstopwords.txt"
> enablePositionIncrements="true"
> />
> <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="1" generateNumberParts="1" catenateWords="1"
> catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.EnglishPorterFilterFactory"
> protected="englishprotwords.txt"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.SynonymFilterFactory"
> synonyms="englishsynonyms.txt" ignoreCase="true" expand="true"/>
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="englishstopwords.txt"/>
> <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="1" generateNumberParts="1" catenateWords="0"
> catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.EnglishPorterFilterFactory"
> protected="englishprotwords.txt"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> </fieldType>
>
> Thanks in advance!!
>
> - Mahendra
>
>
>
>
Re: Exact match search problem
Posted by Chris Hostetter <ho...@fucit.org>.
: I have indexed my data as "custom123, customer, custom" for the
: "UserName" field. I need to search the records for exact match, when I
: am trying to search with UserName:"customer" I am finding the records
: where UserName is custom123 and custom.
:
: As per my understanding solr splits the AlphaNumeric words into sub words
: custom123 => "custom","123"
:
: As per above the above logic when I search for UserName:"customer", it
: shouldn't display the custom123 and custom.
:
: Could you please tell me why it is behaving like that or how I can search for exact match.
"exact match" means differnet things to different people ... based on your
example it seems that you want tokenization on whitespace and punctuation,
but no word-splitting. removing the WordDelimiterFilterFactory from your
field type will probably help ... but you might also want to get rid of
the EnglishPorter and stopwords list too ... it all depends on what your
goal is.
in general, this is what the Analysis Tool (linked to from the admin
screen) is for -- it helps you see what exactly all of your existing
filters are doing so you can decidewhich ones to remove(or alter the
configuration of)
-Hoss