You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Kissue Kissue <ki...@gmail.com> on 2012/06/28 17:10:03 UTC

Is it compulsory to define a tokenizer when defining field types in solr

Hi,

When defining a fieldtype is it compulsory to include a tokenizer in its
definition?

I have a field defined as follows without tokenizer:

<fieldType name="lowercase_pattern" class="solr.TextField"
positionIncrementGap="100">
      <analyzer type="index">
        <filter class="solr.LowerCaseFilterFactory" />
      </analyzer>
      <analyzer type="query">
        <filter class="solr.LowerCaseFilterFactory" />
      </analyzer>
    </fieldType>

Using this field when i try to start up Solr it says the field is not
recognised. But when i change it to the following with tokenizer included
it works:

<fieldType name="lowercase_pattern" class="solr.TextField"
positionIncrementGap="100">
      <analyzer type="index">
        <tokenizer class="solr.KeywordTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory" />
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.KeywordTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory" />
      </analyzer>
    </fieldType>

Thanks.

Re: Is it compulsory to define a tokenizer when defining field types in solr

Posted by Kissue Kissue <ki...@gmail.com>.

Thanks Erick for the clarification.

Cheers!

On Fri, Jun 29, 2012 at 2:08 PM, Erick Erickson <er...@gmail.com>wrote:

> Yes, it's mandatory to define at least one tokenizer (and only one
> tokenizer). If
> you need the whole input treated as one token, you can use
> KeywordTokenizerFactory.
>
> Best
> Erick
>
> On Thu, Jun 28, 2012 at 11:10 AM, Kissue Kissue <ki...@gmail.com>
> wrote:
> > Hi,
> >
> > When defining a fieldtype is it compulsory to include a tokenizer in its
> > definition?
> >
> > I have a field defined as follows without tokenizer:
> >
> > <fieldType name="lowercase_pattern" class="solr.TextField"
> > positionIncrementGap="100">
> >      <analyzer type="index">
> >        <filter class="solr.LowerCaseFilterFactory" />
> >      </analyzer>
> >      <analyzer type="query">
> >        <filter class="solr.LowerCaseFilterFactory" />
> >      </analyzer>
> >    </fieldType>
> >
> > Using this field when i try to start up Solr it says the field is not
> > recognised. But when i change it to the following with tokenizer included
> > it works:
> >
> > <fieldType name="lowercase_pattern" class="solr.TextField"
> > positionIncrementGap="100">
> >      <analyzer type="index">
> >        <tokenizer class="solr.KeywordTokenizerFactory"/>
> >        <filter class="solr.LowerCaseFilterFactory" />
> >      </analyzer>
> >      <analyzer type="query">
> >        <tokenizer class="solr.KeywordTokenizerFactory"/>
> >        <filter class="solr.LowerCaseFilterFactory" />
> >      </analyzer>
> >    </fieldType>
> >
> > Thanks.
>

Re: Is it compulsory to define a tokenizer when defining field types in solr

Posted by Erick Erickson <er...@gmail.com>.

Yes, it's mandatory to define at least one tokenizer (and only one
tokenizer). If
you need the whole input treated as one token, you can use
KeywordTokenizerFactory.

Best
Erick

On Thu, Jun 28, 2012 at 11:10 AM, Kissue Kissue <ki...@gmail.com> wrote:
> Hi,
>
> When defining a fieldtype is it compulsory to include a tokenizer in its
> definition?
>
> I have a field defined as follows without tokenizer:
>
> <fieldType name="lowercase_pattern" class="solr.TextField"
> positionIncrementGap="100">
>      <analyzer type="index">
>        <filter class="solr.LowerCaseFilterFactory" />
>      </analyzer>
>      <analyzer type="query">
>        <filter class="solr.LowerCaseFilterFactory" />
>      </analyzer>
>    </fieldType>
>
> Using this field when i try to start up Solr it says the field is not
> recognised. But when i change it to the following with tokenizer included
> it works:
>
> <fieldType name="lowercase_pattern" class="solr.TextField"
> positionIncrementGap="100">
>      <analyzer type="index">
>        <tokenizer class="solr.KeywordTokenizerFactory"/>
>        <filter class="solr.LowerCaseFilterFactory" />
>      </analyzer>
>      <analyzer type="query">
>        <tokenizer class="solr.KeywordTokenizerFactory"/>
>        <filter class="solr.LowerCaseFilterFactory" />
>      </analyzer>
>    </fieldType>
>
> Thanks.