You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Walter Underwood <wu...@wunderwood.org> on 2017/04/12 16:45:44 UTC

KeywordTokenizer and multiValued field

Does the KeywordTokenizer make each value into a unitary string or does it take the whole list of values and make that a single string?

I really hope it is the former. I can’t find this in the docs (including JavaDocs).

wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/  (my blog)

Re: KeywordTokenizer and multiValued field

Posted by Andrea Gazzarini <gx...@gmail.com>.

Hi Wunder,
I think it's the first option: if you have 3 values then the analyzer 
chain is executed three times.

Andrea

On 12/04/17 18:45, Walter Underwood wrote:
> Does the KeywordTokenizer make each value into a unitary string or does it take the whole list of values and make that a single string?
>
> I really hope it is the former. I can\u2019t find this in the docs (including JavaDocs).
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
>
>

Re: KeywordTokenizer and multiValued field

Posted by Erick Erickson <er...@gmail.com>.

So I have a field named "key" that uses KeywordTokenizer and has
multiValued="true" set. A doc like
<doc>
  <field name="key">val one</field>
  <field name="key">yet another value</field>
  <field name="key">third</field>
</doc>

My field will have exactly three indexed tokens

val one
yet another value
third

Best,
Erick

On Wed, Apr 12, 2017 at 2:38 PM, Ahmet Arslan <io...@yahoo.com.invalid> wrote:
> I don't understand the first option, what is each value? Keyword tokenizer emits single token, analogous to string type.
>
>
>
> On Wednesday, April 12, 2017, 7:45:52 PM GMT+3, Walter Underwood <wu...@wunderwood.org> wrote:
> Does the KeywordTokenizer make each value into a unitary string or does it take the whole list of values and make that a single string?
>
> I really hope it is the former. I can’t find this in the docs (including JavaDocs).
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/  (my blog)

Re: KeywordTokenizer and multiValued field

Posted by Ahmet Arslan <io...@yahoo.com.INVALID>.

I don't understand the first option, what is each value? Keyword tokenizer emits single token, analogous to string type.



On Wednesday, April 12, 2017, 7:45:52 PM GMT+3, Walter Underwood <wu...@wunderwood.org> wrote:
Does the KeywordTokenizer make each value into a unitary string or does it take the whole list of values and make that a single string?

I really hope it is the former. I can’t find this in the docs (including JavaDocs).

wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/  (my blog)