You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by James Norton <jn...@yellowbrix.com> on 2010/09/28 21:31:34 UTC

Using separate Analyzers for querying and indexing.

Hello,

I am migrating from a pure Lucene application to using solr.  For legacy reasons I must support a somewhat obscure query feature: lowercase words in the query should match lowercase or uppercase in the index, while uppercase words in the query should only match uppercase words in the index.

To do this with Lucene we created a custom Analyzer and custom TokenFilter.  During indexing, the custom TokenFilter duplicates uppercase tokens as lowercase ones and sets their offsets to make them appear in same position as the upper case token, i.e., you get two tokens for every uppercase token.  Then at query time a normal (case sensitive) analyzer is used so that lowercase tokens will match either upper or lower, while the uppercase will only match uppercase.

I have looked through the documentation and I see how to specify the Analyzer in the schema.xml file that is used for indexing, but I don't know how to specify that a different Analyzer (the case sensitive one) should be used for queries.

Is this possible?

Thanks,

James

Re: Using separate Analyzers for querying and indexing.

Posted by James Norton <jn...@yellowbrix.com>.
Excellent, exactly what I needed.

Thanks,

James

On Sep 28, 2010, at 4:28 PM, Luke Crouch wrote:

> Yeah. You can specify two analyzers in the same fieldType:
> 
> <fieldType name="..." class="...">
> <analyzer type="index">
> ...
> </analyzer>
> <analyzer type="query">
> ...
> </analyzer>
> </fieldType>
> 
> -L
> 
> On Tue, Sep 28, 2010 at 2:31 PM, James Norton <jn...@yellowbrix.com>wrote:
> 
>> Hello,
>> 
>> I am migrating from a pure Lucene application to using solr.  For legacy
>> reasons I must support a somewhat obscure query feature: lowercase words in
>> the query should match lowercase or uppercase in the index, while uppercase
>> words in the query should only match uppercase words in the index.
>> 
>> To do this with Lucene we created a custom Analyzer and custom TokenFilter.
>> During indexing, the custom TokenFilter duplicates uppercase tokens as
>> lowercase ones and sets their offsets to make them appear in same position
>> as the upper case token, i.e., you get two tokens for every uppercase token.
>> Then at query time a normal (case sensitive) analyzer is used so that
>> lowercase tokens will match either upper or lower, while the uppercase will
>> only match uppercase.
>> 
>> I have looked through the documentation and I see how to specify the
>> Analyzer in the schema.xml file that is used for indexing, but I don't know
>> how to specify that a different Analyzer (the case sensitive one) should be
>> used for queries.
>> 
>> Is this possible?
>> 
>> Thanks,
>> 
>> James

Re: Using separate Analyzers for querying and indexing.

Posted by Luke Crouch <lc...@geek.net>.
Yeah. You can specify two analyzers in the same fieldType:

<fieldType name="..." class="...">
<analyzer type="index">
...
</analyzer>
<analyzer type="query">
...
</analyzer>
</fieldType>

-L

On Tue, Sep 28, 2010 at 2:31 PM, James Norton <jn...@yellowbrix.com>wrote:

> Hello,
>
> I am migrating from a pure Lucene application to using solr.  For legacy
> reasons I must support a somewhat obscure query feature: lowercase words in
> the query should match lowercase or uppercase in the index, while uppercase
> words in the query should only match uppercase words in the index.
>
> To do this with Lucene we created a custom Analyzer and custom TokenFilter.
>  During indexing, the custom TokenFilter duplicates uppercase tokens as
> lowercase ones and sets their offsets to make them appear in same position
> as the upper case token, i.e., you get two tokens for every uppercase token.
>  Then at query time a normal (case sensitive) analyzer is used so that
> lowercase tokens will match either upper or lower, while the uppercase will
> only match uppercase.
>
> I have looked through the documentation and I see how to specify the
> Analyzer in the schema.xml file that is used for indexing, but I don't know
> how to specify that a different Analyzer (the case sensitive one) should be
> used for queries.
>
> Is this possible?
>
> Thanks,
>
> James