You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Rohit Kumar <ro...@gmail.com> on 2013/07/28 03:59:06 UTC

Searching in stopwords

I have a company search which uses stopwords during quezary time. In my
stopwords list i have entries like :

HR
Club
India
Pvt.
Ltd.
....


So if i search for companies like HR Club i get no results. Similarly
search for India HR giving no results. How can i get results in query for
following companies :

1. HR India
2. HR Club
3. HR India Pvt Ltd


I would still want to maintain the above list of stopwords since these
letters occur heavily in company text.

Please guide if i need to change my strategy itself.

<field name="company" type="text_lowercase_whitespace" indexed="true"
stored="true" />



<fieldType name="text_lowercase_whitespace" class="solr.TextField"
positionIncrementGap="100">
           <analyzer type="index">
         	<tokenizer class="solr.WhitespaceTokenizerFactory"/>
     	   	<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt" enablePositionIncrements="true" />
        	<filter class="solr.LowerCaseFilterFactory"/>
		<filter class="solr.PorterStemFilterFactory"/>
   		<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
           </analyzer>
 	   <analyzer type="query">
         	 <tokenizer class="solr.StandardTokenizerFactory"/>
                <filter class="solr.PorterStemFilterFactory"/>
     	   	<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt" enablePositionIncrements="true" />
        	<filter class="solr.LowerCaseFilterFactory"/>
   		<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
           </analyzer>
    </fieldType>



Thanks
Rohit Kumar

Re: Searching in stopwords

Posted by Chris Hostetter <ho...@fucit.org>.
: So if i search for companies like HR Club i get no results. Similarly
: search for India HR giving no results. How can i get results in query for
: following companies :

take a look at the "CommonGramsFilterFactory" and 
"CommonGramsQueryFilterFactory" ... they shold let you remove stopwords, 
but index the pairs of consecutive stop words as single tokens for use at 
query time.

https://lucene.apache.org/core/4_4_0/analyzers-common/org/apache/lucene/analysis/commongrams/package-summary.html



-Hoss

Re: Searching in stopwords

Posted by Jack Krupansky <ja...@basetechnology.com>.
Edismax should be able to handle a query consisting of only query-time stop 
words.

What does your text field type analyzer look like?

-- Jack Krupansky

-----Original Message----- 
From: Rohit Kumar
Sent: Saturday, July 27, 2013 9:59 PM
To: solr-user@lucene.apache.org
Subject: Searching in stopwords

I have a company search which uses stopwords during quezary time. In my
stopwords list i have entries like :

HR
Club
India
Pvt.
Ltd.
....


So if i search for companies like HR Club i get no results. Similarly
search for India HR giving no results. How can i get results in query for
following companies :

1. HR India
2. HR Club
3. HR India Pvt Ltd


I would still want to maintain the above list of stopwords since these
letters occur heavily in company text.

Please guide if i need to change my strategy itself.

<field name="company" type="text_lowercase_whitespace" indexed="true"
stored="true" />



<fieldType name="text_lowercase_whitespace" class="solr.TextField"
positionIncrementGap="100">
           <analyzer type="index">
         <tokenizer class="solr.WhitespaceTokenizerFactory"/>
        <filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt" enablePositionIncrements="true" />
        <filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.PorterStemFilterFactory"/>
   <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
           </analyzer>
     <analyzer type="query">
         <tokenizer class="solr.StandardTokenizerFactory"/>
                <filter class="solr.PorterStemFilterFactory"/>
        <filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt" enablePositionIncrements="true" />
        <filter class="solr.LowerCaseFilterFactory"/>
   <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
           </analyzer>
    </fieldType>



Thanks
Rohit Kumar