You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Alistair Young <al...@uhi.ac.uk> on 2015/06/17 12:15:31 UTC
suggester returning stems instead of whole words
I was wondering if there's a way to get the suggester to return whole words. Instead of returning 'technology' , 'temperature' and 'tutorial', it's returning 'technolog' , 'temperatur' and 'tutori'
using this config:
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggest</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFactory</str>
<str name="field">dc.subject</str>
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
</lst>
</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/suggest">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggest</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">10</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
thanks,
Alistair
--
mov eax,1
mov ebx,0
int 80h
Re: suggester returning stems instead of whole words
Posted by Alistair Young <al...@uhi.ac.uk>.
yep, 4.3.1. The API changed after that so it’s finding the time to rewrite
the entire backend that uses it
On 17/06/2015 16:55, "Shalin Shekhar Mangar" <sh...@gmail.com>
wrote:
>You must be using an old version of Solr. Since Solr 4.8 and beyond,
>the <fields> and <types> tags have been deprecated and you can place
>the field and field type definitions anywhere in the schema.xml.
>
>See http://issues.apache.org/jira/browse/SOLR-5228
>
>On Wed, Jun 17, 2015 at 9:09 PM, Alistair Young
><al...@uhi.ac.uk> wrote:
>> working in a tiny tmux window does have some disadvantages, such as
>>losing
>> one’s place in the file! the subject_autocomplete definition wasn’t
>>inside
>> <fields>. Now that it is, everything is working. thanks for listening
>>
>> Alistair
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>
>>
>>
>>
>> On 17/06/2015 15:17, "Alistair Young" <al...@uhi.ac.uk> wrote:
>>
>>>looking at the schema browser, subject_autocomplete has a type of
>>>text_en
>>>rather than text_auto and all the terms are stemmed. Its contents are
>>>the
>>>same as the one it was copied from, dc.subject, which is text_en and
>>>stemmed.
>>>
>>>On 17/06/2015 14:58, "Erick Erickson" <er...@gmail.com> wrote:
>>>
>>>>Hmmm, shouldn't be happening that way. Spellcheck is supposed to be
>>>>looking at indexed terms. If you go into the admin/schema browser
>>>>page and look at the new field, what are the terms in the index? They
>>>>shouldn't be stemmed.
>>>>
>>>>And I always get confused where this
>>>> <str name="spellcheck.dictionary">suggest</str>
>>>>is supposed to point. Do you have any other component named "suggest"
>>>>that you might be picking up?
>>>>
>>>>Best,
>>>>Erick
>>>>
>>>>On Wed, Jun 17, 2015 at 6:50 AM, Alistair Young
>>>><al...@uhi.ac.uk> wrote:
>>>>> yep did both of those things. Getting the same results as using
>>>>>dc.subject
>>>>>
>>>>> On 17/06/2015 14:44, "Shalin Shekhar Mangar" <sh...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>>Did you change the SpellCheckComponent's configuration to use
>>>>>>subject_autocomplete instead of dc.subject? After you made that
>>>>>>change, did you invoke spellcheck.build=true to re-build the
>>>>>>spellcheck index?
>>>>>>
>>>>>>On Wed, Jun 17, 2015 at 7:06 PM, Alistair Young
>>>>>><al...@uhi.ac.uk> wrote:
>>>>>>> copyField doesn¹t seem to fix the suggestion stemming. Copying the
>>>>>>>field
>>>>>>> to another field of this type:
>>>>>>>
>>>>>>> <field name="subject_autocomplete" type="text_auto" indexed="true"
>>>>>>> stored="true" multiValued="false" />
>>>>>>>
>>>>>>> <copyField source="dc.subject" dest="subject_autocomplete" />
>>>>>>>
>>>>>>>
>>>>>>> <fieldType class="solr.TextField" name="text_auto"
>>>>>>> positionIncrementGap="100">
>>>>>>> <analyzer>
>>>>>>> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>>>>>>> <filter class="solr.WordDelimiterFilterFactory"
>>>>>>>generateWordParts="1"
>>>>>>> generateNumberParts="1" catenateWords="1" catenateNumbers="1"
>>>>>>> catenateAll="0" splitOnCaseChange="1"/>
>>>>>>> <filter class="solr.LowerCaseFilterFactory"/>
>>>>>>> </analyzer>
>>>>>>> </fieldType>
>>>>>>>
>>>>>>>
>>>>>>> but I¹m still getting stemmed suggestions after rebuilding the
>>>>>>>index.
>>>>>>>
>>>>>>> Alistair
>>>>>>>
>>>>>>> --
>>>>>>> mov eax,1
>>>>>>> mov ebx,0
>>>>>>> int 80h
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 17/06/2015 11:28, "Alistair Young" <al...@uhi.ac.uk>
>>>>>>>wrote:
>>>>>>>
>>>>>>>>ah looks like I need to use copyField to get a non stemmed version
>>>>>>>>of
>>>>>>>>the
>>>>>>>>suggester field
>>>>>>>>
>>>>>>>>Alistair
>>>>>>>>
>>>>>>>>--
>>>>>>>>mov eax,1
>>>>>>>>mov ebx,0
>>>>>>>>int 80h
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>On 17/06/2015 11:15, "Alistair Young" <al...@uhi.ac.uk>
>>>>>>>>wrote:
>>>>>>>>
>>>>>>>>>I was wondering if there's a way to get the suggester to return
>>>>>>>>>whole
>>>>>>>>>words. Instead of returning 'technology' , 'temperature' and
>>>>>>>>>'tutorial',
>>>>>>>>>it's returning 'technolog' , 'temperatur' and 'tutori'
>>>>>>>>>
>>>>>>>>>using this config:
>>>>>>>>>
>>>>>>>>><searchComponent class="solr.SpellCheckComponent" name="suggest">
>>>>>>>>> <lst name="spellchecker">
>>>>>>>>> <str name="name">suggest</str>
>>>>>>>>> <str
>>>>>>>>>name="classname">org.apache.solr.spelling.suggest.Suggester</str>
>>>>>>>>> <str
>>>>>>>>>name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupF
>>>>>>>>>ac
>>>>>>>>>t
>>>>>>>>>or
>>>>>>>>>y<
>>>>>>>>>/
>>>>>>>>>str>
>>>>>>>>> <str name="field">dc.subject</str>
>>>>>>>>> <float name="threshold">0.005</float>
>>>>>>>>> <str name="buildOnCommit">true</str>
>>>>>>>>> </lst>
>>>>>>>>> </searchComponent>
>>>>>>>>> <requestHandler
>>>>>>>>>class="org.apache.solr.handler.component.SearchHandler"
>>>>>>>>>name="/suggest">
>>>>>>>>> <lst name="defaults">
>>>>>>>>> <str name="spellcheck">true</str>
>>>>>>>>> <str name="spellcheck.dictionary">suggest</str>
>>>>>>>>> <str name="spellcheck.onlyMorePopular">true</str>
>>>>>>>>> <str name="spellcheck.count">10</str>
>>>>>>>>> <str name="spellcheck.collate">true</str>
>>>>>>>>> </lst>
>>>>>>>>> <arr name="components">
>>>>>>>>> <str>suggest</str>
>>>>>>>>> </arr>
>>>>>>>>> </requestHandler>
>>>>>>>>>
>>>>>>>>>thanks,
>>>>>>>>>
>>>>>>>>>Alistair
>>>>>>>>>
>>>>>>>>>--
>>>>>>>>>mov eax,1
>>>>>>>>>mov ebx,0
>>>>>>>>>int 80h
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>--
>>>>>>Regards,
>>>>>>Shalin Shekhar Mangar.
>>>>>
>>>
>>
>
>
>
>--
>Regards,
>Shalin Shekhar Mangar.
Re: suggester returning stems instead of whole words
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
You must be using an old version of Solr. Since Solr 4.8 and beyond,
the <fields> and <types> tags have been deprecated and you can place
the field and field type definitions anywhere in the schema.xml.
See http://issues.apache.org/jira/browse/SOLR-5228
On Wed, Jun 17, 2015 at 9:09 PM, Alistair Young
<al...@uhi.ac.uk> wrote:
> working in a tiny tmux window does have some disadvantages, such as losing
> one’s place in the file! the subject_autocomplete definition wasn’t inside
> <fields>. Now that it is, everything is working. thanks for listening
>
> Alistair
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 17/06/2015 15:17, "Alistair Young" <al...@uhi.ac.uk> wrote:
>
>>looking at the schema browser, subject_autocomplete has a type of text_en
>>rather than text_auto and all the terms are stemmed. Its contents are the
>>same as the one it was copied from, dc.subject, which is text_en and
>>stemmed.
>>
>>On 17/06/2015 14:58, "Erick Erickson" <er...@gmail.com> wrote:
>>
>>>Hmmm, shouldn't be happening that way. Spellcheck is supposed to be
>>>looking at indexed terms. If you go into the admin/schema browser
>>>page and look at the new field, what are the terms in the index? They
>>>shouldn't be stemmed.
>>>
>>>And I always get confused where this
>>> <str name="spellcheck.dictionary">suggest</str>
>>>is supposed to point. Do you have any other component named "suggest"
>>>that you might be picking up?
>>>
>>>Best,
>>>Erick
>>>
>>>On Wed, Jun 17, 2015 at 6:50 AM, Alistair Young
>>><al...@uhi.ac.uk> wrote:
>>>> yep did both of those things. Getting the same results as using
>>>>dc.subject
>>>>
>>>> On 17/06/2015 14:44, "Shalin Shekhar Mangar" <sh...@gmail.com>
>>>> wrote:
>>>>
>>>>>Did you change the SpellCheckComponent's configuration to use
>>>>>subject_autocomplete instead of dc.subject? After you made that
>>>>>change, did you invoke spellcheck.build=true to re-build the
>>>>>spellcheck index?
>>>>>
>>>>>On Wed, Jun 17, 2015 at 7:06 PM, Alistair Young
>>>>><al...@uhi.ac.uk> wrote:
>>>>>> copyField doesn¹t seem to fix the suggestion stemming. Copying the
>>>>>>field
>>>>>> to another field of this type:
>>>>>>
>>>>>> <field name="subject_autocomplete" type="text_auto" indexed="true"
>>>>>> stored="true" multiValued="false" />
>>>>>>
>>>>>> <copyField source="dc.subject" dest="subject_autocomplete" />
>>>>>>
>>>>>>
>>>>>> <fieldType class="solr.TextField" name="text_auto"
>>>>>> positionIncrementGap="100">
>>>>>> <analyzer>
>>>>>> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>>>>>> <filter class="solr.WordDelimiterFilterFactory"
>>>>>>generateWordParts="1"
>>>>>> generateNumberParts="1" catenateWords="1" catenateNumbers="1"
>>>>>> catenateAll="0" splitOnCaseChange="1"/>
>>>>>> <filter class="solr.LowerCaseFilterFactory"/>
>>>>>> </analyzer>
>>>>>> </fieldType>
>>>>>>
>>>>>>
>>>>>> but I¹m still getting stemmed suggestions after rebuilding the index.
>>>>>>
>>>>>> Alistair
>>>>>>
>>>>>> --
>>>>>> mov eax,1
>>>>>> mov ebx,0
>>>>>> int 80h
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 17/06/2015 11:28, "Alistair Young" <al...@uhi.ac.uk>
>>>>>>wrote:
>>>>>>
>>>>>>>ah looks like I need to use copyField to get a non stemmed version of
>>>>>>>the
>>>>>>>suggester field
>>>>>>>
>>>>>>>Alistair
>>>>>>>
>>>>>>>--
>>>>>>>mov eax,1
>>>>>>>mov ebx,0
>>>>>>>int 80h
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>On 17/06/2015 11:15, "Alistair Young" <al...@uhi.ac.uk>
>>>>>>>wrote:
>>>>>>>
>>>>>>>>I was wondering if there's a way to get the suggester to return
>>>>>>>>whole
>>>>>>>>words. Instead of returning 'technology' , 'temperature' and
>>>>>>>>'tutorial',
>>>>>>>>it's returning 'technolog' , 'temperatur' and 'tutori'
>>>>>>>>
>>>>>>>>using this config:
>>>>>>>>
>>>>>>>><searchComponent class="solr.SpellCheckComponent" name="suggest">
>>>>>>>> <lst name="spellchecker">
>>>>>>>> <str name="name">suggest</str>
>>>>>>>> <str
>>>>>>>>name="classname">org.apache.solr.spelling.suggest.Suggester</str>
>>>>>>>> <str
>>>>>>>>name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFac
>>>>>>>>t
>>>>>>>>or
>>>>>>>>y<
>>>>>>>>/
>>>>>>>>str>
>>>>>>>> <str name="field">dc.subject</str>
>>>>>>>> <float name="threshold">0.005</float>
>>>>>>>> <str name="buildOnCommit">true</str>
>>>>>>>> </lst>
>>>>>>>> </searchComponent>
>>>>>>>> <requestHandler
>>>>>>>>class="org.apache.solr.handler.component.SearchHandler"
>>>>>>>>name="/suggest">
>>>>>>>> <lst name="defaults">
>>>>>>>> <str name="spellcheck">true</str>
>>>>>>>> <str name="spellcheck.dictionary">suggest</str>
>>>>>>>> <str name="spellcheck.onlyMorePopular">true</str>
>>>>>>>> <str name="spellcheck.count">10</str>
>>>>>>>> <str name="spellcheck.collate">true</str>
>>>>>>>> </lst>
>>>>>>>> <arr name="components">
>>>>>>>> <str>suggest</str>
>>>>>>>> </arr>
>>>>>>>> </requestHandler>
>>>>>>>>
>>>>>>>>thanks,
>>>>>>>>
>>>>>>>>Alistair
>>>>>>>>
>>>>>>>>--
>>>>>>>>mov eax,1
>>>>>>>>mov ebx,0
>>>>>>>>int 80h
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>--
>>>>>Regards,
>>>>>Shalin Shekhar Mangar.
>>>>
>>
>
--
Regards,
Shalin Shekhar Mangar.
Re: suggester returning stems instead of whole words
Posted by Alistair Young <al...@uhi.ac.uk>.
working in a tiny tmux window does have some disadvantages, such as losing
one’s place in the file! the subject_autocomplete definition wasn’t inside
<fields>. Now that it is, everything is working. thanks for listening
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 17/06/2015 15:17, "Alistair Young" <al...@uhi.ac.uk> wrote:
>looking at the schema browser, subject_autocomplete has a type of text_en
>rather than text_auto and all the terms are stemmed. Its contents are the
>same as the one it was copied from, dc.subject, which is text_en and
>stemmed.
>
>On 17/06/2015 14:58, "Erick Erickson" <er...@gmail.com> wrote:
>
>>Hmmm, shouldn't be happening that way. Spellcheck is supposed to be
>>looking at indexed terms. If you go into the admin/schema browser
>>page and look at the new field, what are the terms in the index? They
>>shouldn't be stemmed.
>>
>>And I always get confused where this
>> <str name="spellcheck.dictionary">suggest</str>
>>is supposed to point. Do you have any other component named "suggest"
>>that you might be picking up?
>>
>>Best,
>>Erick
>>
>>On Wed, Jun 17, 2015 at 6:50 AM, Alistair Young
>><al...@uhi.ac.uk> wrote:
>>> yep did both of those things. Getting the same results as using
>>>dc.subject
>>>
>>> On 17/06/2015 14:44, "Shalin Shekhar Mangar" <sh...@gmail.com>
>>> wrote:
>>>
>>>>Did you change the SpellCheckComponent's configuration to use
>>>>subject_autocomplete instead of dc.subject? After you made that
>>>>change, did you invoke spellcheck.build=true to re-build the
>>>>spellcheck index?
>>>>
>>>>On Wed, Jun 17, 2015 at 7:06 PM, Alistair Young
>>>><al...@uhi.ac.uk> wrote:
>>>>> copyField doesn¹t seem to fix the suggestion stemming. Copying the
>>>>>field
>>>>> to another field of this type:
>>>>>
>>>>> <field name="subject_autocomplete" type="text_auto" indexed="true"
>>>>> stored="true" multiValued="false" />
>>>>>
>>>>> <copyField source="dc.subject" dest="subject_autocomplete" />
>>>>>
>>>>>
>>>>> <fieldType class="solr.TextField" name="text_auto"
>>>>> positionIncrementGap="100">
>>>>> <analyzer>
>>>>> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>>>>> <filter class="solr.WordDelimiterFilterFactory"
>>>>>generateWordParts="1"
>>>>> generateNumberParts="1" catenateWords="1" catenateNumbers="1"
>>>>> catenateAll="0" splitOnCaseChange="1"/>
>>>>> <filter class="solr.LowerCaseFilterFactory"/>
>>>>> </analyzer>
>>>>> </fieldType>
>>>>>
>>>>>
>>>>> but I¹m still getting stemmed suggestions after rebuilding the index.
>>>>>
>>>>> Alistair
>>>>>
>>>>> --
>>>>> mov eax,1
>>>>> mov ebx,0
>>>>> int 80h
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 17/06/2015 11:28, "Alistair Young" <al...@uhi.ac.uk>
>>>>>wrote:
>>>>>
>>>>>>ah looks like I need to use copyField to get a non stemmed version of
>>>>>>the
>>>>>>suggester field
>>>>>>
>>>>>>Alistair
>>>>>>
>>>>>>--
>>>>>>mov eax,1
>>>>>>mov ebx,0
>>>>>>int 80h
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>On 17/06/2015 11:15, "Alistair Young" <al...@uhi.ac.uk>
>>>>>>wrote:
>>>>>>
>>>>>>>I was wondering if there's a way to get the suggester to return
>>>>>>>whole
>>>>>>>words. Instead of returning 'technology' , 'temperature' and
>>>>>>>'tutorial',
>>>>>>>it's returning 'technolog' , 'temperatur' and 'tutori'
>>>>>>>
>>>>>>>using this config:
>>>>>>>
>>>>>>><searchComponent class="solr.SpellCheckComponent" name="suggest">
>>>>>>> <lst name="spellchecker">
>>>>>>> <str name="name">suggest</str>
>>>>>>> <str
>>>>>>>name="classname">org.apache.solr.spelling.suggest.Suggester</str>
>>>>>>> <str
>>>>>>>name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFac
>>>>>>>t
>>>>>>>or
>>>>>>>y<
>>>>>>>/
>>>>>>>str>
>>>>>>> <str name="field">dc.subject</str>
>>>>>>> <float name="threshold">0.005</float>
>>>>>>> <str name="buildOnCommit">true</str>
>>>>>>> </lst>
>>>>>>> </searchComponent>
>>>>>>> <requestHandler
>>>>>>>class="org.apache.solr.handler.component.SearchHandler"
>>>>>>>name="/suggest">
>>>>>>> <lst name="defaults">
>>>>>>> <str name="spellcheck">true</str>
>>>>>>> <str name="spellcheck.dictionary">suggest</str>
>>>>>>> <str name="spellcheck.onlyMorePopular">true</str>
>>>>>>> <str name="spellcheck.count">10</str>
>>>>>>> <str name="spellcheck.collate">true</str>
>>>>>>> </lst>
>>>>>>> <arr name="components">
>>>>>>> <str>suggest</str>
>>>>>>> </arr>
>>>>>>> </requestHandler>
>>>>>>>
>>>>>>>thanks,
>>>>>>>
>>>>>>>Alistair
>>>>>>>
>>>>>>>--
>>>>>>>mov eax,1
>>>>>>>mov ebx,0
>>>>>>>int 80h
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>--
>>>>Regards,
>>>>Shalin Shekhar Mangar.
>>>
>
Re: suggester returning stems instead of whole words
Posted by Alistair Young <al...@uhi.ac.uk>.
looking at the schema browser, subject_autocomplete has a type of text_en
rather than text_auto and all the terms are stemmed. Its contents are the
same as the one it was copied from, dc.subject, which is text_en and
stemmed.
On 17/06/2015 14:58, "Erick Erickson" <er...@gmail.com> wrote:
>Hmmm, shouldn't be happening that way. Spellcheck is supposed to be
>looking at indexed terms. If you go into the admin/schema browser
>page and look at the new field, what are the terms in the index? They
>shouldn't be stemmed.
>
>And I always get confused where this
> <str name="spellcheck.dictionary">suggest</str>
>is supposed to point. Do you have any other component named "suggest"
>that you might be picking up?
>
>Best,
>Erick
>
>On Wed, Jun 17, 2015 at 6:50 AM, Alistair Young
><al...@uhi.ac.uk> wrote:
>> yep did both of those things. Getting the same results as using
>>dc.subject
>>
>> On 17/06/2015 14:44, "Shalin Shekhar Mangar" <sh...@gmail.com>
>> wrote:
>>
>>>Did you change the SpellCheckComponent's configuration to use
>>>subject_autocomplete instead of dc.subject? After you made that
>>>change, did you invoke spellcheck.build=true to re-build the
>>>spellcheck index?
>>>
>>>On Wed, Jun 17, 2015 at 7:06 PM, Alistair Young
>>><al...@uhi.ac.uk> wrote:
>>>> copyField doesn¹t seem to fix the suggestion stemming. Copying the
>>>>field
>>>> to another field of this type:
>>>>
>>>> <field name="subject_autocomplete" type="text_auto" indexed="true"
>>>> stored="true" multiValued="false" />
>>>>
>>>> <copyField source="dc.subject" dest="subject_autocomplete" />
>>>>
>>>>
>>>> <fieldType class="solr.TextField" name="text_auto"
>>>> positionIncrementGap="100">
>>>> <analyzer>
>>>> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>>>> <filter class="solr.WordDelimiterFilterFactory"
>>>>generateWordParts="1"
>>>> generateNumberParts="1" catenateWords="1" catenateNumbers="1"
>>>> catenateAll="0" splitOnCaseChange="1"/>
>>>> <filter class="solr.LowerCaseFilterFactory"/>
>>>> </analyzer>
>>>> </fieldType>
>>>>
>>>>
>>>> but I¹m still getting stemmed suggestions after rebuilding the index.
>>>>
>>>> Alistair
>>>>
>>>> --
>>>> mov eax,1
>>>> mov ebx,0
>>>> int 80h
>>>>
>>>>
>>>>
>>>>
>>>> On 17/06/2015 11:28, "Alistair Young" <al...@uhi.ac.uk>
>>>>wrote:
>>>>
>>>>>ah looks like I need to use copyField to get a non stemmed version of
>>>>>the
>>>>>suggester field
>>>>>
>>>>>Alistair
>>>>>
>>>>>--
>>>>>mov eax,1
>>>>>mov ebx,0
>>>>>int 80h
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>On 17/06/2015 11:15, "Alistair Young" <al...@uhi.ac.uk>
>>>>>wrote:
>>>>>
>>>>>>I was wondering if there's a way to get the suggester to return whole
>>>>>>words. Instead of returning 'technology' , 'temperature' and
>>>>>>'tutorial',
>>>>>>it's returning 'technolog' , 'temperatur' and 'tutori'
>>>>>>
>>>>>>using this config:
>>>>>>
>>>>>><searchComponent class="solr.SpellCheckComponent" name="suggest">
>>>>>> <lst name="spellchecker">
>>>>>> <str name="name">suggest</str>
>>>>>> <str
>>>>>>name="classname">org.apache.solr.spelling.suggest.Suggester</str>
>>>>>> <str
>>>>>>name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFact
>>>>>>or
>>>>>>y<
>>>>>>/
>>>>>>str>
>>>>>> <str name="field">dc.subject</str>
>>>>>> <float name="threshold">0.005</float>
>>>>>> <str name="buildOnCommit">true</str>
>>>>>> </lst>
>>>>>> </searchComponent>
>>>>>> <requestHandler
>>>>>>class="org.apache.solr.handler.component.SearchHandler"
>>>>>>name="/suggest">
>>>>>> <lst name="defaults">
>>>>>> <str name="spellcheck">true</str>
>>>>>> <str name="spellcheck.dictionary">suggest</str>
>>>>>> <str name="spellcheck.onlyMorePopular">true</str>
>>>>>> <str name="spellcheck.count">10</str>
>>>>>> <str name="spellcheck.collate">true</str>
>>>>>> </lst>
>>>>>> <arr name="components">
>>>>>> <str>suggest</str>
>>>>>> </arr>
>>>>>> </requestHandler>
>>>>>>
>>>>>>thanks,
>>>>>>
>>>>>>Alistair
>>>>>>
>>>>>>--
>>>>>>mov eax,1
>>>>>>mov ebx,0
>>>>>>int 80h
>>>>>
>>>>
>>>
>>>
>>>
>>>--
>>>Regards,
>>>Shalin Shekhar Mangar.
>>
Re: suggester returning stems instead of whole words
Posted by Erick Erickson <er...@gmail.com>.
Hmmm, shouldn't be happening that way. Spellcheck is supposed to be
looking at indexed terms. If you go into the admin/schema browser
page and look at the new field, what are the terms in the index? They
shouldn't be stemmed.
And I always get confused where this
<str name="spellcheck.dictionary">suggest</str>
is supposed to point. Do you have any other component named "suggest"
that you might be picking up?
Best,
Erick
On Wed, Jun 17, 2015 at 6:50 AM, Alistair Young
<al...@uhi.ac.uk> wrote:
> yep did both of those things. Getting the same results as using dc.subject
>
> On 17/06/2015 14:44, "Shalin Shekhar Mangar" <sh...@gmail.com>
> wrote:
>
>>Did you change the SpellCheckComponent's configuration to use
>>subject_autocomplete instead of dc.subject? After you made that
>>change, did you invoke spellcheck.build=true to re-build the
>>spellcheck index?
>>
>>On Wed, Jun 17, 2015 at 7:06 PM, Alistair Young
>><al...@uhi.ac.uk> wrote:
>>> copyField doesn¹t seem to fix the suggestion stemming. Copying the field
>>> to another field of this type:
>>>
>>> <field name="subject_autocomplete" type="text_auto" indexed="true"
>>> stored="true" multiValued="false" />
>>>
>>> <copyField source="dc.subject" dest="subject_autocomplete" />
>>>
>>>
>>> <fieldType class="solr.TextField" name="text_auto"
>>> positionIncrementGap="100">
>>> <analyzer>
>>> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>>> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1"
>>> generateNumberParts="1" catenateWords="1" catenateNumbers="1"
>>> catenateAll="0" splitOnCaseChange="1"/>
>>> <filter class="solr.LowerCaseFilterFactory"/>
>>> </analyzer>
>>> </fieldType>
>>>
>>>
>>> but I¹m still getting stemmed suggestions after rebuilding the index.
>>>
>>> Alistair
>>>
>>> --
>>> mov eax,1
>>> mov ebx,0
>>> int 80h
>>>
>>>
>>>
>>>
>>> On 17/06/2015 11:28, "Alistair Young" <al...@uhi.ac.uk> wrote:
>>>
>>>>ah looks like I need to use copyField to get a non stemmed version of
>>>>the
>>>>suggester field
>>>>
>>>>Alistair
>>>>
>>>>--
>>>>mov eax,1
>>>>mov ebx,0
>>>>int 80h
>>>>
>>>>
>>>>
>>>>
>>>>On 17/06/2015 11:15, "Alistair Young" <al...@uhi.ac.uk> wrote:
>>>>
>>>>>I was wondering if there's a way to get the suggester to return whole
>>>>>words. Instead of returning 'technology' , 'temperature' and
>>>>>'tutorial',
>>>>>it's returning 'technolog' , 'temperatur' and 'tutori'
>>>>>
>>>>>using this config:
>>>>>
>>>>><searchComponent class="solr.SpellCheckComponent" name="suggest">
>>>>> <lst name="spellchecker">
>>>>> <str name="name">suggest</str>
>>>>> <str
>>>>>name="classname">org.apache.solr.spelling.suggest.Suggester</str>
>>>>> <str
>>>>>name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFactor
>>>>>y<
>>>>>/
>>>>>str>
>>>>> <str name="field">dc.subject</str>
>>>>> <float name="threshold">0.005</float>
>>>>> <str name="buildOnCommit">true</str>
>>>>> </lst>
>>>>> </searchComponent>
>>>>> <requestHandler
>>>>>class="org.apache.solr.handler.component.SearchHandler"
>>>>>name="/suggest">
>>>>> <lst name="defaults">
>>>>> <str name="spellcheck">true</str>
>>>>> <str name="spellcheck.dictionary">suggest</str>
>>>>> <str name="spellcheck.onlyMorePopular">true</str>
>>>>> <str name="spellcheck.count">10</str>
>>>>> <str name="spellcheck.collate">true</str>
>>>>> </lst>
>>>>> <arr name="components">
>>>>> <str>suggest</str>
>>>>> </arr>
>>>>> </requestHandler>
>>>>>
>>>>>thanks,
>>>>>
>>>>>Alistair
>>>>>
>>>>>--
>>>>>mov eax,1
>>>>>mov ebx,0
>>>>>int 80h
>>>>
>>>
>>
>>
>>
>>--
>>Regards,
>>Shalin Shekhar Mangar.
>
Re: suggester returning stems instead of whole words
Posted by Alistair Young <al...@uhi.ac.uk>.
yep did both of those things. Getting the same results as using dc.subject
On 17/06/2015 14:44, "Shalin Shekhar Mangar" <sh...@gmail.com>
wrote:
>Did you change the SpellCheckComponent's configuration to use
>subject_autocomplete instead of dc.subject? After you made that
>change, did you invoke spellcheck.build=true to re-build the
>spellcheck index?
>
>On Wed, Jun 17, 2015 at 7:06 PM, Alistair Young
><al...@uhi.ac.uk> wrote:
>> copyField doesn¹t seem to fix the suggestion stemming. Copying the field
>> to another field of this type:
>>
>> <field name="subject_autocomplete" type="text_auto" indexed="true"
>> stored="true" multiValued="false" />
>>
>> <copyField source="dc.subject" dest="subject_autocomplete" />
>>
>>
>> <fieldType class="solr.TextField" name="text_auto"
>> positionIncrementGap="100">
>> <analyzer>
>> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1"
>> generateNumberParts="1" catenateWords="1" catenateNumbers="1"
>> catenateAll="0" splitOnCaseChange="1"/>
>> <filter class="solr.LowerCaseFilterFactory"/>
>> </analyzer>
>> </fieldType>
>>
>>
>> but I¹m still getting stemmed suggestions after rebuilding the index.
>>
>> Alistair
>>
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>
>>
>>
>>
>> On 17/06/2015 11:28, "Alistair Young" <al...@uhi.ac.uk> wrote:
>>
>>>ah looks like I need to use copyField to get a non stemmed version of
>>>the
>>>suggester field
>>>
>>>Alistair
>>>
>>>--
>>>mov eax,1
>>>mov ebx,0
>>>int 80h
>>>
>>>
>>>
>>>
>>>On 17/06/2015 11:15, "Alistair Young" <al...@uhi.ac.uk> wrote:
>>>
>>>>I was wondering if there's a way to get the suggester to return whole
>>>>words. Instead of returning 'technology' , 'temperature' and
>>>>'tutorial',
>>>>it's returning 'technolog' , 'temperatur' and 'tutori'
>>>>
>>>>using this config:
>>>>
>>>><searchComponent class="solr.SpellCheckComponent" name="suggest">
>>>> <lst name="spellchecker">
>>>> <str name="name">suggest</str>
>>>> <str
>>>>name="classname">org.apache.solr.spelling.suggest.Suggester</str>
>>>> <str
>>>>name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFactor
>>>>y<
>>>>/
>>>>str>
>>>> <str name="field">dc.subject</str>
>>>> <float name="threshold">0.005</float>
>>>> <str name="buildOnCommit">true</str>
>>>> </lst>
>>>> </searchComponent>
>>>> <requestHandler
>>>>class="org.apache.solr.handler.component.SearchHandler"
>>>>name="/suggest">
>>>> <lst name="defaults">
>>>> <str name="spellcheck">true</str>
>>>> <str name="spellcheck.dictionary">suggest</str>
>>>> <str name="spellcheck.onlyMorePopular">true</str>
>>>> <str name="spellcheck.count">10</str>
>>>> <str name="spellcheck.collate">true</str>
>>>> </lst>
>>>> <arr name="components">
>>>> <str>suggest</str>
>>>> </arr>
>>>> </requestHandler>
>>>>
>>>>thanks,
>>>>
>>>>Alistair
>>>>
>>>>--
>>>>mov eax,1
>>>>mov ebx,0
>>>>int 80h
>>>
>>
>
>
>
>--
>Regards,
>Shalin Shekhar Mangar.
Re: suggester returning stems instead of whole words
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Did you change the SpellCheckComponent's configuration to use
subject_autocomplete instead of dc.subject? After you made that
change, did you invoke spellcheck.build=true to re-build the
spellcheck index?
On Wed, Jun 17, 2015 at 7:06 PM, Alistair Young
<al...@uhi.ac.uk> wrote:
> copyField doesn¹t seem to fix the suggestion stemming. Copying the field
> to another field of this type:
>
> <field name="subject_autocomplete" type="text_auto" indexed="true"
> stored="true" multiValued="false" />
>
> <copyField source="dc.subject" dest="subject_autocomplete" />
>
>
> <fieldType class="solr.TextField" name="text_auto"
> positionIncrementGap="100">
> <analyzer>
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1"
> generateNumberParts="1" catenateWords="1" catenateNumbers="1"
> catenateAll="0" splitOnCaseChange="1"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> </analyzer>
> </fieldType>
>
>
> but I¹m still getting stemmed suggestions after rebuilding the index.
>
> Alistair
>
> --
> mov eax,1
> mov ebx,0
> int 80h
>
>
>
>
> On 17/06/2015 11:28, "Alistair Young" <al...@uhi.ac.uk> wrote:
>
>>ah looks like I need to use copyField to get a non stemmed version of the
>>suggester field
>>
>>Alistair
>>
>>--
>>mov eax,1
>>mov ebx,0
>>int 80h
>>
>>
>>
>>
>>On 17/06/2015 11:15, "Alistair Young" <al...@uhi.ac.uk> wrote:
>>
>>>I was wondering if there's a way to get the suggester to return whole
>>>words. Instead of returning 'technology' , 'temperature' and 'tutorial',
>>>it's returning 'technolog' , 'temperatur' and 'tutori'
>>>
>>>using this config:
>>>
>>><searchComponent class="solr.SpellCheckComponent" name="suggest">
>>> <lst name="spellchecker">
>>> <str name="name">suggest</str>
>>> <str
>>>name="classname">org.apache.solr.spelling.suggest.Suggester</str>
>>> <str
>>>name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFactory<
>>>/
>>>str>
>>> <str name="field">dc.subject</str>
>>> <float name="threshold">0.005</float>
>>> <str name="buildOnCommit">true</str>
>>> </lst>
>>> </searchComponent>
>>> <requestHandler class="org.apache.solr.handler.component.SearchHandler"
>>>name="/suggest">
>>> <lst name="defaults">
>>> <str name="spellcheck">true</str>
>>> <str name="spellcheck.dictionary">suggest</str>
>>> <str name="spellcheck.onlyMorePopular">true</str>
>>> <str name="spellcheck.count">10</str>
>>> <str name="spellcheck.collate">true</str>
>>> </lst>
>>> <arr name="components">
>>> <str>suggest</str>
>>> </arr>
>>> </requestHandler>
>>>
>>>thanks,
>>>
>>>Alistair
>>>
>>>--
>>>mov eax,1
>>>mov ebx,0
>>>int 80h
>>
>
--
Regards,
Shalin Shekhar Mangar.
Re: suggester returning stems instead of whole words
Posted by Alistair Young <al...@uhi.ac.uk>.
copyField doesn¹t seem to fix the suggestion stemming. Copying the field
to another field of this type:
<field name="subject_autocomplete" type="text_auto" indexed="true"
stored="true" multiValued="false" />
<copyField source="dc.subject" dest="subject_autocomplete" />
<fieldType class="solr.TextField" name="text_auto"
positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1"
generateNumberParts="1" catenateWords="1" catenateNumbers="1"
catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
but I¹m still getting stemmed suggestions after rebuilding the index.
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 17/06/2015 11:28, "Alistair Young" <al...@uhi.ac.uk> wrote:
>ah looks like I need to use copyField to get a non stemmed version of the
>suggester field
>
>Alistair
>
>--
>mov eax,1
>mov ebx,0
>int 80h
>
>
>
>
>On 17/06/2015 11:15, "Alistair Young" <al...@uhi.ac.uk> wrote:
>
>>I was wondering if there's a way to get the suggester to return whole
>>words. Instead of returning 'technology' , 'temperature' and 'tutorial',
>>it's returning 'technolog' , 'temperatur' and 'tutori'
>>
>>using this config:
>>
>><searchComponent class="solr.SpellCheckComponent" name="suggest">
>> <lst name="spellchecker">
>> <str name="name">suggest</str>
>> <str
>>name="classname">org.apache.solr.spelling.suggest.Suggester</str>
>> <str
>>name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFactory<
>>/
>>str>
>> <str name="field">dc.subject</str>
>> <float name="threshold">0.005</float>
>> <str name="buildOnCommit">true</str>
>> </lst>
>> </searchComponent>
>> <requestHandler class="org.apache.solr.handler.component.SearchHandler"
>>name="/suggest">
>> <lst name="defaults">
>> <str name="spellcheck">true</str>
>> <str name="spellcheck.dictionary">suggest</str>
>> <str name="spellcheck.onlyMorePopular">true</str>
>> <str name="spellcheck.count">10</str>
>> <str name="spellcheck.collate">true</str>
>> </lst>
>> <arr name="components">
>> <str>suggest</str>
>> </arr>
>> </requestHandler>
>>
>>thanks,
>>
>>Alistair
>>
>>--
>>mov eax,1
>>mov ebx,0
>>int 80h
>
Re: suggester returning stems instead of whole words
Posted by Alistair Young <al...@uhi.ac.uk>.
ah looks like I need to use copyField to get a non stemmed version of the
suggester field
Alistair
--
mov eax,1
mov ebx,0
int 80h
On 17/06/2015 11:15, "Alistair Young" <al...@uhi.ac.uk> wrote:
>I was wondering if there's a way to get the suggester to return whole
>words. Instead of returning 'technology' , 'temperature' and 'tutorial',
>it's returning 'technolog' , 'temperatur' and 'tutori'
>
>using this config:
>
><searchComponent class="solr.SpellCheckComponent" name="suggest">
> <lst name="spellchecker">
> <str name="name">suggest</str>
> <str
>name="classname">org.apache.solr.spelling.suggest.Suggester</str>
> <str
>name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFactory</
>str>
> <str name="field">dc.subject</str>
> <float name="threshold">0.005</float>
> <str name="buildOnCommit">true</str>
> </lst>
> </searchComponent>
> <requestHandler class="org.apache.solr.handler.component.SearchHandler"
>name="/suggest">
> <lst name="defaults">
> <str name="spellcheck">true</str>
> <str name="spellcheck.dictionary">suggest</str>
> <str name="spellcheck.onlyMorePopular">true</str>
> <str name="spellcheck.count">10</str>
> <str name="spellcheck.collate">true</str>
> </lst>
> <arr name="components">
> <str>suggest</str>
> </arr>
> </requestHandler>
>
>thanks,
>
>Alistair
>
>--
>mov eax,1
>mov ebx,0
>int 80h