You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Thomas Michael Engelke <th...@posteo.de> on 2015/06/29 18:22:21 UTC
Questions regarding autosuggest (Solr 5.2.1)
A friend and I are trying to develop some software using Solr in the
background, and with that comes alot of changes. We're used to older
versions (4.3 and below). We especially have problems with the
autosuggest feature.
This is the field definition (schema.xml) for our autosuggest field:
<field name="autosuggest" type="autosuggest" indexed="true"
stored="true" required="false" multiValued="true" />
...
<copyField source="name" dest="autosuggest" />
...
<fieldType name="autosuggest" class="solr.TextField"
positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.WordDelimiterFilterFactory" splitOnCaseChange="0"
splitOnNumerics="1" generateWordParts="1" generateNumberParts="1"
catenateWords="1" catenateNumbers="0" catenateAll="0"
preserveOriginal="0"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" words="stopwords.txt"
ignoreCase="true" enablePositionIncrements="true" format="snowball"/>
<filter class="solr.DictionaryCompoundWordTokenFilterFactory"
dictionary="dictionary.txt" minWordSize="5" minSubwordSize="3"
maxSubwordSize="30" onlyLongestMatch="false"/>
<filter class="solr.GermanNormalizationFilterFactory"/>
<filter class="solr.SnowballPorterFilterFactory" language="German2"
protected="protwords.txt"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="2"
maxGramSize="30"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.WordDelimiterFilterFactory" splitOnCaseChange="0"
splitOnNumerics="1" generateWordParts="1" generateNumberParts="1"
catenateWords="1" catenateNumbers="0" catenateAll="0"
preserveOriginal="0"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" words="stopwords.txt"
ignoreCase="true" enablePositionIncrements="true" format="snowball"/>
<filter class="solr.GermanNormalizationFilterFactory"/>
<filter class="solr.SnowballPorterFilterFactory" language="German2"
protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
Afterwards, we defined an autosuggest component to use this field, like
this (solrconfig.xml):
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">FuzzyLookupFactory</str>
<str name="storeDir">suggester_fuzzy_dir</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">suggest</str>
<str name="suggestAnalyzerFieldType">"autosuggest"</str>
<str name="buildOnStartup">false</str>
<str name="buildOnCommit">false</str>
</lst>
</searchComponent>
And add a requesthandler to test out the functionality:
<requestHandler name="/suggesthandler" class="solr.SearchHandler"
startup="lazy" >
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
<str name="suggest.dictionary">mySuggester</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
However, trying to start the core that has this configuration, a long
exception occurs, telling us this:
Error in configuration: "autosuggest" is not defined in the schema
Now, that seems to be wrong. Any idea how to fix that?
Re: Questions regarding autosuggest (Solr 5.2.1)
Posted by Alessandro Benedetti <be...@gmail.com>.
I would like to add some consideration if possible.
I find the field type really hard analysed, are you sure is this ok with
your suggestions requirement ?
Usually is better to keep the field for suggestion as less analysed as
possible and then play with the different type of suggesters.
If you notice any additional problem, we can discuss through that, if not ,
it is ok !
Cheers
2015-06-30 13:48 GMT+01:00 Erick Erickson <er...@gmail.com>:
> Pesky computers, they keep doing exactly what I tell 'em to do, not
> what I mean ;)
>
> I'll open a JIRA for making Solr DWIM-compliant, Do What I Mean ;) ;)
>
> On Tue, Jun 30, 2015 at 4:17 AM, Thomas Michael Engelke
> <th...@posteo.de> wrote:
> > God damn. Thank you.
> >
> > *ashamed*
> >
> > Am 30.06.2015 00:21 schrieb Erick Erickson:
> >
> >> Try not putting it in double quotes?
> >>
> >> Best,
> >> Erick
> >>
> >> On Mon, Jun 29, 2015 at 12:22 PM, Thomas Michael Engelke
> >> <th...@posteo.de> wrote:
> >>
> >>> A friend and I are trying to develop some software using Solr in the
> background, and with that comes alot of changes. We're used to older
> versions (4.3 and below). We especially have problems with the autosuggest
> feature. This is the field definition (schema.xml) for our autosuggest
> field: <field name="autosuggest" type="autosuggest" indexed="true"
> stored="true" required="false" multiValued="true" /> ... <copyField
> source="name" dest="autosuggest" /> ... <fieldType name="autosuggest"
> class="solr.TextField" positionIncrementGap="100"> <analyzer type="index">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter
> class="solr.WordDelimiterFilterFactory" splitOnCaseChange="0"
> splitOnNumerics="1" generateWordParts="1" generateNumberParts="1"
> catenateWords="1" catenateNumbers="0" catenateAll="0"
> preserveOriginal="0"/> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.StopFilterFactory" words="stopwords.txt"
> ignoreCase="true" enablePositionIncrements="true"
> > format="snowball"/> <filter
> class="solr.DictionaryCompoundWordTokenFilterFactory"
> dictionary="dictionary.txt" minWordSize="5" minSubwordSize="3"
> maxSubwordSize="30" onlyLongestMatch="false"/> <filter
> class="solr.GermanNormalizationFilterFactory"/> <filter
> class="solr.SnowballPorterFilterFactory" language="German2"
> protected="protwords.txt"/> <filter class="solr.EdgeNGramFilterFactory"
> minGramSize="2" maxGramSize="30"/> <filter
> class="solr.RemoveDuplicatesTokenFilterFactory"/> </analyzer> <analyzer
> type="query"> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter
> class="solr.WordDelimiterFilterFactory" splitOnCaseChange="0"
> splitOnNumerics="1" generateWordParts="1" generateNumberParts="1"
> catenateWords="1" catenateNumbers="0" catenateAll="0"
> preserveOriginal="0"/> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.StopFilterFactory" words="stopwords.txt"
> ignoreCase="true" enablePositionIncrements="true" format="snowball"/>
> <filter
> > class="solr.GermanNormalizationFilterFactory"/> <filter
> class="solr.SnowballPorterFilterFactory" language="German2"
> protected="protwords.txt"/> <filter
> class="solr.RemoveDuplicatesTokenFilterFactory"/> </analyzer> </fieldType>
> Afterwards, we defined an autosuggest component to use this field, like
> this (solrconfig.xml): <searchComponent name="suggest"
> class="solr.SuggestComponent"> <lst name="suggester"> <str
> name="name">mySuggester</str> <str
> name="lookupImpl">FuzzyLookupFactory</str> <str
> name="storeDir">suggester_fuzzy_dir</str> <str
> name="dictionaryImpl">DocumentDictionaryFactory</str> <str
> name="field">suggest</str> <str
> name="suggestAnalyzerFieldType">"autosuggest"</str> <str
> name="buildOnStartup">false</str> <str name="buildOnCommit">false</str>
> </lst> </searchComponent> And add a requesthandler to test out the
> functionality: <requestHandler name="/suggesthandler"
> class="solr.SearchHandler" startup="lazy" > <lst name="defaults"> <str
> name="suggest">true</str> <str
> > name="suggest.count">10</str> <str
> name="suggest.dictionary">mySuggester</str> </lst> <arr name="components">
> <str>suggest</str> </arr> </requestHandler> However, trying to start the
> core that has this configuration, a long exception occurs, telling us this:
> Error in configuration: "autosuggest" is not defined in the schema Now,
> that seems to be wrong. Any idea how to fix that?
> >
>
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Questions regarding autosuggest (Solr 5.2.1)
Posted by Erick Erickson <er...@gmail.com>.
Pesky computers, they keep doing exactly what I tell 'em to do, not
what I mean ;)
I'll open a JIRA for making Solr DWIM-compliant, Do What I Mean ;) ;)
On Tue, Jun 30, 2015 at 4:17 AM, Thomas Michael Engelke
<th...@posteo.de> wrote:
> God damn. Thank you.
>
> *ashamed*
>
> Am 30.06.2015 00:21 schrieb Erick Erickson:
>
>> Try not putting it in double quotes?
>>
>> Best,
>> Erick
>>
>> On Mon, Jun 29, 2015 at 12:22 PM, Thomas Michael Engelke
>> <th...@posteo.de> wrote:
>>
>>> A friend and I are trying to develop some software using Solr in the background, and with that comes alot of changes. We're used to older versions (4.3 and below). We especially have problems with the autosuggest feature. This is the field definition (schema.xml) for our autosuggest field: <field name="autosuggest" type="autosuggest" indexed="true" stored="true" required="false" multiValued="true" /> ... <copyField source="name" dest="autosuggest" /> ... <fieldType name="autosuggest" class="solr.TextField" positionIncrementGap="100"> <analyzer type="index"> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter class="solr.WordDelimiterFilterFactory" splitOnCaseChange="0" splitOnNumerics="1" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="0" catenateAll="0" preserveOriginal="0"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true" enablePositionIncrements="true"
> format="snowball"/> <filter class="solr.DictionaryCompoundWordTokenFilterFactory" dictionary="dictionary.txt" minWordSize="5" minSubwordSize="3" maxSubwordSize="30" onlyLongestMatch="false"/> <filter class="solr.GermanNormalizationFilterFactory"/> <filter class="solr.SnowballPorterFilterFactory" language="German2" protected="protwords.txt"/> <filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="30"/> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/> </analyzer> <analyzer type="query"> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter class="solr.WordDelimiterFilterFactory" splitOnCaseChange="0" splitOnNumerics="1" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="0" catenateAll="0" preserveOriginal="0"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true" enablePositionIncrements="true" format="snowball"/> <filter
> class="solr.GermanNormalizationFilterFactory"/> <filter class="solr.SnowballPorterFilterFactory" language="German2" protected="protwords.txt"/> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/> </analyzer> </fieldType> Afterwards, we defined an autosuggest component to use this field, like this (solrconfig.xml): <searchComponent name="suggest" class="solr.SuggestComponent"> <lst name="suggester"> <str name="name">mySuggester</str> <str name="lookupImpl">FuzzyLookupFactory</str> <str name="storeDir">suggester_fuzzy_dir</str> <str name="dictionaryImpl">DocumentDictionaryFactory</str> <str name="field">suggest</str> <str name="suggestAnalyzerFieldType">"autosuggest"</str> <str name="buildOnStartup">false</str> <str name="buildOnCommit">false</str> </lst> </searchComponent> And add a requesthandler to test out the functionality: <requestHandler name="/suggesthandler" class="solr.SearchHandler" startup="lazy" > <lst name="defaults"> <str name="suggest">true</str> <str
> name="suggest.count">10</str> <str name="suggest.dictionary">mySuggester</str> </lst> <arr name="components"> <str>suggest</str> </arr> </requestHandler> However, trying to start the core that has this configuration, a long exception occurs, telling us this: Error in configuration: "autosuggest" is not defined in the schema Now, that seems to be wrong. Any idea how to fix that?
>
Re: Questions regarding autosuggest (Solr 5.2.1)
Posted by Thomas Michael Engelke <th...@posteo.de>.
God damn. Thank you.
*ashamed*
Am 30.06.2015 00:21 schrieb Erick Erickson:
> Try not putting it in double quotes?
>
> Best,
> Erick
>
> On Mon, Jun 29, 2015 at 12:22 PM, Thomas Michael Engelke
> <th...@posteo.de> wrote:
>
>> A friend and I are trying to develop some software using Solr in the background, and with that comes alot of changes. We're used to older versions (4.3 and below). We especially have problems with the autosuggest feature. This is the field definition (schema.xml) for our autosuggest field: <field name="autosuggest" type="autosuggest" indexed="true" stored="true" required="false" multiValued="true" /> ... <copyField source="name" dest="autosuggest" /> ... <fieldType name="autosuggest" class="solr.TextField" positionIncrementGap="100"> <analyzer type="index"> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter class="solr.WordDelimiterFilterFactory" splitOnCaseChange="0" splitOnNumerics="1" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="0" catenateAll="0" preserveOriginal="0"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true" enablePositionIncrements="true"
format="snowball"/> <filter class="solr.DictionaryCompoundWordTokenFilterFactory" dictionary="dictionary.txt" minWordSize="5" minSubwordSize="3" maxSubwordSize="30" onlyLongestMatch="false"/> <filter class="solr.GermanNormalizationFilterFactory"/> <filter class="solr.SnowballPorterFilterFactory" language="German2" protected="protwords.txt"/> <filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="30"/> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/> </analyzer> <analyzer type="query"> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter class="solr.WordDelimiterFilterFactory" splitOnCaseChange="0" splitOnNumerics="1" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="0" catenateAll="0" preserveOriginal="0"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true" enablePositionIncrements="true" format="snowball"/> <filter
class="solr.GermanNormalizationFilterFactory"/> <filter class="solr.SnowballPorterFilterFactory" language="German2" protected="protwords.txt"/> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/> </analyzer> </fieldType> Afterwards, we defined an autosuggest component to use this field, like this (solrconfig.xml): <searchComponent name="suggest" class="solr.SuggestComponent"> <lst name="suggester"> <str name="name">mySuggester</str> <str name="lookupImpl">FuzzyLookupFactory</str> <str name="storeDir">suggester_fuzzy_dir</str> <str name="dictionaryImpl">DocumentDictionaryFactory</str> <str name="field">suggest</str> <str name="suggestAnalyzerFieldType">"autosuggest"</str> <str name="buildOnStartup">false</str> <str name="buildOnCommit">false</str> </lst> </searchComponent> And add a requesthandler to test out the functionality: <requestHandler name="/suggesthandler" class="solr.SearchHandler" startup="lazy" > <lst name="defaults"> <str name="suggest">true</str> <str
name="suggest.count">10</str> <str name="suggest.dictionary">mySuggester</str> </lst> <arr name="components"> <str>suggest</str> </arr> </requestHandler> However, trying to start the core that has this configuration, a long exception occurs, telling us this: Error in configuration: "autosuggest" is not defined in the schema Now, that seems to be wrong. Any idea how to fix that?
Re: Questions regarding autosuggest (Solr 5.2.1)
Posted by Erick Erickson <er...@gmail.com>.
Try not putting it in double quotes?
Best,
Erick
On Mon, Jun 29, 2015 at 12:22 PM, Thomas Michael Engelke
<th...@posteo.de> wrote:
>
>
> A friend and I are trying to develop some software using Solr in the
> background, and with that comes alot of changes. We're used to older
> versions (4.3 and below). We especially have problems with the
> autosuggest feature.
>
> This is the field definition (schema.xml) for our autosuggest field:
>
> <field name="autosuggest" type="autosuggest" indexed="true"
> stored="true" required="false" multiValued="true" />
> ...
> <copyField source="name" dest="autosuggest" />
> ...
> <fieldType name="autosuggest" class="solr.TextField"
> positionIncrementGap="100">
> <analyzer type="index">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.WordDelimiterFilterFactory" splitOnCaseChange="0"
> splitOnNumerics="1" generateWordParts="1" generateNumberParts="1"
> catenateWords="1" catenateNumbers="0" catenateAll="0"
> preserveOriginal="0"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.StopFilterFactory" words="stopwords.txt"
> ignoreCase="true" enablePositionIncrements="true" format="snowball"/>
> <filter class="solr.DictionaryCompoundWordTokenFilterFactory"
> dictionary="dictionary.txt" minWordSize="5" minSubwordSize="3"
> maxSubwordSize="30" onlyLongestMatch="false"/>
> <filter class="solr.GermanNormalizationFilterFactory"/>
> <filter class="solr.SnowballPorterFilterFactory" language="German2"
> protected="protwords.txt"/>
> <filter class="solr.EdgeNGramFilterFactory" minGramSize="2"
> maxGramSize="30"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.WordDelimiterFilterFactory" splitOnCaseChange="0"
> splitOnNumerics="1" generateWordParts="1" generateNumberParts="1"
> catenateWords="1" catenateNumbers="0" catenateAll="0"
> preserveOriginal="0"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.StopFilterFactory" words="stopwords.txt"
> ignoreCase="true" enablePositionIncrements="true" format="snowball"/>
> <filter class="solr.GermanNormalizationFilterFactory"/>
> <filter class="solr.SnowballPorterFilterFactory" language="German2"
> protected="protwords.txt"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> </fieldType>
>
> Afterwards, we defined an autosuggest component to use this field, like
> this (solrconfig.xml):
>
> <searchComponent name="suggest" class="solr.SuggestComponent">
> <lst name="suggester">
> <str name="name">mySuggester</str>
> <str name="lookupImpl">FuzzyLookupFactory</str>
> <str name="storeDir">suggester_fuzzy_dir</str>
> <str name="dictionaryImpl">DocumentDictionaryFactory</str>
> <str name="field">suggest</str>
> <str name="suggestAnalyzerFieldType">"autosuggest"</str>
> <str name="buildOnStartup">false</str>
> <str name="buildOnCommit">false</str>
> </lst>
> </searchComponent>
>
> And add a requesthandler to test out the functionality:
>
> <requestHandler name="/suggesthandler" class="solr.SearchHandler"
> startup="lazy" >
> <lst name="defaults">
> <str name="suggest">true</str>
> <str name="suggest.count">10</str>
> <str name="suggest.dictionary">mySuggester</str>
> </lst>
> <arr name="components">
> <str>suggest</str>
> </arr>
> </requestHandler>
>
> However, trying to start the core that has this configuration, a long
> exception occurs, telling us this:
>
> Error in configuration: "autosuggest" is not defined in the schema
>
> Now, that seems to be wrong. Any idea how to fix that?