Question on using SynonymFilterFactory

[Prasanna Ranganathan <pr...@netflix.com>]

Hi,

 I have a question regarding the usage of the SynonymFilterFactory at index
time.

 We have a document field with attributes indexed=true, stored=true
(multiValued not explicitly set in schema.xml) and also use the
SynonymFilterFactory as one of the index analyzer filters for that field. I
examined (using Luke) one of the documents in the index that should have a
synonym match expansion for this field but do not see the additional values
for that field. The field seems to have only the original value and does not
show the synonym matches.

The relevant excerpts from schema.xml are as follows:

<fieldtype name="text" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
  <tokenizer class="solr.WhitespaceTokenizerFactory"/>
  <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
ignoreCase="true" expand="true"/>
   ....

<field name="name" type="text" indexed="true" stored="true"/>

I am curious whether not setting the multiValued attribute to true for this
field results in the synonym match expansions not being indexed. If they are
indexed then how can I check it using Luke? Also, what is the default
setting of the multiValued attribute when not explicitly specified for a
field in schema.xml? I looked at the available documentation but am unable
to figure it out. 

Kindly help.

Regards,

Prasanna.

Re: Question on using SynonymFilterFactory

[Prasanna Ranganathan <pr...@netflix.com>]

On 9/18/09 3:17 PM, "AHMET ARSLAN" <io...@yahoo.com> wrote:

>>  We have a document field with attributes indexed=true,
>> stored=true
>> (multiValued not explicitly set in schema.xml) and also use
>> the
>> SynonymFilterFactory as one of the index analyzer filters
>> for that field. I
>> examined (using Luke) one of the documents in the index
>> that should have a
>> synonym match expansion for this field but do not see the
>> additional values
>> for that field. The field seems to have only the original
>> value and does not
>> show the synonym matches.
>> 
> 
> I don't know much about Luke. May be Luke is not showing tokens at the same
> position?
> 
>> I am curious whether not setting the multiValued attribute
>> to true for this
>> field results in the synonym match expansions not being
>> indexed. If they are
>> indexed then how can I check it using Luke? Also, what is
>> the default
>> setting of the multiValued attribute when not explicitly
>> specified for a
>> field in schema.xml? I looked at the available
>> documentation but am unable
>> to figure it out.
> 
> You can see/check your synonyms in /admin/analysis.jsp page. Output of the
> tokenizer and token filters.
> 

 I did verify it with the admin analysis tool and it checked out. Still, I
was a bit doubtful since Luke did not show the injected synonyms for that
field. It could very well be something to do with Luke.

> Synonym injection works fine with multiValued=false which is default behavior.
> 
> "1.1: multiValued attribute introduced, false by default" from schema.xml
> 
> Hope this helps.
> 

 Thanks a lot for the response. Much appreciated.

Regards,

Prasanna.

Re: Question on using SynonymFilterFactory

[AHMET ARSLAN <io...@yahoo.com>]

>  I have a question regarding the usage of the
> SynonymFilterFactory at index
> time.
> 
>  We have a document field with attributes indexed=true,
> stored=true
> (multiValued not explicitly set in schema.xml) and also use
> the
> SynonymFilterFactory as one of the index analyzer filters
> for that field. I
> examined (using Luke) one of the documents in the index
> that should have a
> synonym match expansion for this field but do not see the
> additional values
> for that field. The field seems to have only the original
> value and does not
> show the synonym matches.
> 

I don't know much about Luke. May be Luke is not showing tokens at the same position?

> I am curious whether not setting the multiValued attribute
> to true for this
> field results in the synonym match expansions not being
> indexed. If they are
> indexed then how can I check it using Luke? Also, what is
> the default
> setting of the multiValued attribute when not explicitly
> specified for a
> field in schema.xml? I looked at the available
> documentation but am unable
> to figure it out. 

You can see/check your synonyms in /admin/analysis.jsp page. Output of the tokenizer and token filters.

Synonym injection works fine with multiValued=false which is default behavior.

"1.1: multiValued attribute introduced, false by default" from schema.xml

Hope this helps.