dismax and stopwords (was Re: dismax and long phrases)

[Jon Drukman <jd...@gmail.com>]

Norberto Meijome wrote:
> On Tue, 07 Oct 2008 09:27:30 -0700
> Jon Drukman <jd...@gmail.com> wrote:
> 
>>> Yep, you can "fake" it by only using fieldsets (qf) that have a 
>>> consistent set of stopwords.  
>> does that mean changing the query or changing the schema?
> 
> Jon,
> - you change schema.xml to define which type each field is. The fieldType says whether you have stopwords or not.
> - you change solrconfig.xml to define which fields will dismax query on.
> 
> i dont think you should have to change your query.

i got it to work.  the solution is:

add a new field to the schema without stopwords, i use the following type:

   <fieldType name="text_ws" class="solr.TextField" 
positionIncrementGap="100">
       <analyzer>
         <tokenizer class="solr.WhitespaceTokenizerFactory"/>
         <filter class="solr.LowerCaseFilterFactory"/>
       </analyzer>
     </fieldType>


then use copyField to copy the stopworded version to a second, 
non-stopworded field.  add the non-stopword field to the dismax qf and 
pf fields.  in this example, the stopword field is name and the 
non-stopword field is name_text:

  <str name="qf">
         name^1.5 name_text^1.8 description^1.0 tags^0.5 location^0.6 
user_name^0.4 misc^0.3 group_name^1.5
      </str>
      <str name="pf">
         name^1.5 name_text^1.8 description^1.0 group_name^1.5
      </str>


restart solr and reindex everything.  it now works.

thanks for all the help!

-jsd-