You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Max Scheffler <ma...@freiheit.com> on 2007/10/25 16:44:29 UTC

prefix-search ingnores the lowerCaseFilter

Hi,

I want to perform a prefix-search which ignores cases. To do this I 
created a fielType called suggest:

<fieldType name="suggest" class="solr.TextField" positionIncrementGap="100">
   <analyzer type="index">
     <tokenizer class="solr.StandardTokenizerFactory"/>
     <filter class="solr.LowerCaseFilterFactory"/>
     <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
   </analyzer>
   <analyzer type="query">
     <tokenizer class="solr.StandardTokenizerFactory"/>
     <filter class="solr.LowerCaseFilterFactory"/>
   </analyzer>
</fieldType>

Entrys (terms) could be 'foo', 'bar'...

A request like

http://localhost:8983/solr/select/?rows=0&facet=true&q=*:*&facet.field=suggest&facet.prefix=f

returns things like

<lst name="facet_counts">
   <lst name="facet_queries"/>
   <lst name="facet_fields">
     <lst name="suggest">
       <int name="foo">12</int>
     </lst>
   </lst>
</lst>

But a request like
http://localhost:8983/solr/select/?rows=0&facet=true&q=*:*&facet.field=suggest&facet.prefix=F

returns just:

<lst name="facet_counts">
   <lst name="facet_queries"/>
   <lst name="facet_fields">
     <lst name="suggest"/>
   </lst>
</lst>

That's not what I've expected, cause the field-definition contains a 
LowerCaseFilter.

Is it possible that the prefix-processing ignores the filters?

Max

Re: prefix-search ingnores the lowerCaseFilter

Posted by Martin Grotzke <ma...@javakaffee.de>.
On Mon, 2007-10-29 at 13:31 -0400, Yonik Seeley wrote:
> On 10/29/07, Martin Grotzke <ma...@javakaffee.de> wrote:
> > On Thu, 2007-10-25 at 10:48 -0400, Yonik Seeley wrote:
> > > On 10/25/07, Max Scheffler <ma...@freiheit.com> wrote:
> > > > Is it possible that the prefix-processing ignores the filters?
> > >
> > > Yes, It's a known limitation that we haven't worked out a fix for yet.
> > > The issue is that you can't just run the prefix through the filters
> > > because of things like stop words, stemming, minimum length filters,
> > > etc.
> >
> > What about not having only "facet.prefix" but additionally
> > "facet.filtered.prefix" that runs the prefix through the filters?
> > Would that be possible?
> 
> The underlying issue remains - it's not safe to treat the prefix like
> any other word when running it through the filters.
Yes, definitely the user that uses this feature should know what it
does - but at least there would be the possibility to run the prefix
through e.g. a LowerCaseFilter. Finally the user knows what filters
he has configured. E.g. if you only want an ignore-case prefix test,
s.th. like a facet.filtered.prefix would be really valuable.

Cheers,
Martin


> 
> -Yonik
> 


Re: prefix-search ingnores the lowerCaseFilter

Posted by Yonik Seeley <yo...@apache.org>.
On 10/29/07, Martin Grotzke <ma...@javakaffee.de> wrote:
> On Thu, 2007-10-25 at 10:48 -0400, Yonik Seeley wrote:
> > On 10/25/07, Max Scheffler <ma...@freiheit.com> wrote:
> > > Is it possible that the prefix-processing ignores the filters?
> >
> > Yes, It's a known limitation that we haven't worked out a fix for yet.
> > The issue is that you can't just run the prefix through the filters
> > because of things like stop words, stemming, minimum length filters,
> > etc.
>
> What about not having only "facet.prefix" but additionally
> "facet.filtered.prefix" that runs the prefix through the filters?
> Would that be possible?

The underlying issue remains - it's not safe to treat the prefix like
any other word when running it through the filters.

-Yonik

Re: prefix-search ingnores the lowerCaseFilter

Posted by Martin Grotzke <ma...@javakaffee.de>.
On Thu, 2007-10-25 at 10:48 -0400, Yonik Seeley wrote:
> On 10/25/07, Max Scheffler <ma...@freiheit.com> wrote:
> > Is it possible that the prefix-processing ignores the filters?
> 
> Yes, It's a known limitation that we haven't worked out a fix for yet.
> The issue is that you can't just run the prefix through the filters
> because of things like stop words, stemming, minimum length filters,
> etc.
What about not having only "facet.prefix" but additionally
"facet.filtered.prefix" that runs the prefix through the filters?
Would that be possible?

Cheers,
Martin

> 
> -Yonik
> 


Re: prefix-search ingnores the lowerCaseFilter

Posted by Yonik Seeley <yo...@apache.org>.
On 10/25/07, Max Scheffler <ma...@freiheit.com> wrote:
> Is it possible that the prefix-processing ignores the filters?

Yes, It's a known limitation that we haven't worked out a fix for yet.
The issue is that you can't just run the prefix through the filters
because of things like stop words, stemming, minimum length filters,
etc.

-Yonik