You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Scott Vanderbilt <li...@datagenic.com> on 2016/08/25 23:41:32 UTC
Re: Solr Suggester no results
I'm having the exact same problem the O.P. describes from his email back
in May, but my configuration dose not have the same defect his had. So I
am at a loss to understand why my suggest queries are returning no results.
Here is my config:
Relevant bits from schema.xml:
--------------------------------------
<field name="roadName" type="string" indexed="true" stored="true"
multiValued="true"/>
<field name="commentaryP" type="text_general" indexed="true"
stored="true" multiValued="true"/>
...
<field name="text_suggest" type="textSuggest" indexed="true"
stored="true" multiValued="true" />
...
<copyField source="roadName" dest="text_suggest" />
<copyField source="commentaryP" dest="text_suggest" />
...
<fieldType class="solr.TextField" name="textSuggest"
positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
Relevant bits from solrconfig.xml:
--------------------------------------
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="suggest.dictionary">mySuggester</str>
<str name="suggest">true</str>
<str name="suggest.count">10</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
...
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">AnalyzingInfixLookupFactory</str>
<str name="indexPath">suggester_infix_dir</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">text_suggest</str>
<str name="suggestAnalyzerFieldType">string</str>
<str name="buildOnStartup">false</str>
<str name="buildOnCommit">false</str>
</lst>
</searchComponent>
I build the dictionary like this:
http://example.com:8983/solr/rrib/suggest?suggest.build=true
and get back this response:
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">964</int>
</lst>
<str name="command">build</str>
</response>
I then attempt a query like this:
http://example.com:8983/solr/rrib/suggest?q=re
and get back this response:
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">37</int>
</lst>
<lst name="suggest">
<lst name="mySuggester">
<lst name="re">
<int name="numFound">0</int>
<arr name="suggestions"/>
</lst>
</lst>
</lst>
</response>
In Solr's Admin interface, I see the following under OTHER->suggest on
Plugins/Stats page:
class:suggest
description:Suggester component
src:
version:6.1.0
stats:
mySuggester: SolrSuggester [ name=mySuggester,
sourceLocation=null, storeDir=,
lookupImpl=AnalyzingInfixLookupFactory,
dictionaryImpl=DocumentDictionaryFactory, sizeInBytes=6795 ]
totalSizeInBytes: 6795
The value of 6,795 bytes seems pretty small to me for a repository of
403 XML files containing about 1.5 MB of mark-up. Perhaps that is a clue
that the dictionary has not been fully populated, which probably
explains the empty result sets, but I cannot figure out why
Any assistance would be gratefully received.
Thank you.
On 5/6/2016 9:42 AM, Erick Erickson wrote:
> First off, kudos for providing the details, that really helps!
>
> The root of your problem is that your suggest field has stored="false".
> DocumentDictionaryFactory reads through all the
> docs in your corpus, extracts the stored data and puts it in the FST. Since
> you don't have any stored data your FST is...er...minimal.
>
> I'd also add
> <str name="storeDir">suggester_fuzzy_dir</str>
> to the searchComponent. You'll find the FST on disk in that directory where it
> can be read next time Solr starts up. It is also helpful for figuring out
> whether there are suggestions to be had.
>
> And a minor nit, you probably don't want to specify suggest.dictionary
> in your query,
> that's already specified in your config.
>
> And it looks like you're alive to the fact that with that setup
> capitalization matters
> as does the fact that these suggestions be matched from the beginning of the
> field...
>
> Best,
> Erick
>
> On Thu, May 5, 2016 at 1:05 AM, Grigoris Iliopoulos
> <gr...@gmail.com> wrote:
>> Hi there,
>>
>> I want to use the Solr suggester component for city names. I have the
>> following settings:
>> schema.xml
>>
>> Field definition
>>
>> <fieldType class="solr.TextField" name="textSuggest" positionIncrementGap="100">
>> <analyzer>
>> <tokenizer class="solr.StandardTokenizerFactory"/>
>> <filter class="solr.StandardFilterFactory"/>
>> <filter class="solr.LowerCaseFilterFactory"/>
>> </analyzer>
>> </fieldType>
>>
>> The field i want to apply the suggester on
>>
>> <field name="city" type="string" indexed="true" stored="false"/>
>>
>> The copy field
>>
>> <copyField source="city" dest="citySuggest"/>
>>
>> The field
>>
>> <field name="citySuggest" type="textSuggest" stored="false" indexed="true" />
>>
>> solr-config.xml
>>
>> <requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
>> <lst name="defaults">
>> <str name="suggest">true</str>
>> <str name="suggest.count">10</str>
>> <str name="suggest.dictionary">mySuggester</str>
>> </lst>
>> <arr name="components">
>> <str>suggest</str>
>> </arr>
>> </requestHandler>
>>
>>
>>
>> <searchComponent name="suggest" class="solr.SuggestComponent">
>> <lst name="suggester">
>> <str name="name">mySuggester</str>
>> <str name="lookupImpl">FuzzyLookupFactory</str>
>> <str name="dictionaryImpl">DocumentDictionaryFactory</str>
>> <str name="field">citySuggest</str>
>> <str name="suggestAnalyzerFieldType">string</str>
>> </lst>
>> </searchComponent>
>>
>> Then i run
>>
>> http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath&suggest.build=true
>>
>> to build the suggest component
>>
>> Finally i run
>>
>> http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath
>>
>> but i get an empty result set
>>
>> {"responseHeader":{"status":0,"QTime":0},"suggest":{"mySuggester":{"Ath":{"numFound":0,"suggestions":[]}}}}
>>
>> Are there any obvious mistakes? Any thoughts?
>
Re: Solr Suggester no results
Posted by Scott Vanderbilt <li...@datagenic.com>.
Bradley:
You're a bloody genius!
That's exactly what I needed to make it work.
For the sake of the archives, after modifying the solrconfig.xml as
indicated and rebuilding the suggester dictionary, the queries started
to kick back results like crazy.
For what it's worth, I'm running Solr 6.1.0.
Thanks so much.
On 8/25/2016 6:48 PM, Bradley Belyeu wrote:
> Scott, I\u2019m fairly new to suggesters having just recently built my first one. But where my configuration differs from yours is on this line:
>
> <str name="suggestAnalyzerFieldType">string</str>
>
> I used the field type name that I had defined instead like:
>
> <str name="suggestAnalyzerFieldType"> textSuggest </str>
>
> I\u2019m not certain that would help, but I can\u2019t see where your config is much different form mine elsewhere.
> What version of Solr are you running?
>
>
> On 8/25/16, 6:41 PM, "Scott Vanderbilt" <li...@datagenic.com> wrote:
>
> I'm having the exact same problem the O.P. describes from his email back
> in May, but my configuration dose not have the same defect his had. So I
> am at a loss to understand why my suggest queries are returning no results.
>
> Here is my config:
>
> Relevant bits from schema.xml:
> --------------------------------------
> <field name="roadName" type="string" indexed="true" stored="true"
> multiValued="true"/>
> <field name="commentaryP" type="text_general" indexed="true"
> stored="true" multiValued="true"/>
> ...
> <field name="text_suggest" type="textSuggest" indexed="true"
> stored="true" multiValued="true" />
> ...
> <copyField source="roadName" dest="text_suggest" />
> <copyField source="commentaryP" dest="text_suggest" />
> ...
> <fieldType class="solr.TextField" name="textSuggest"
> positionIncrementGap="100">
> <analyzer>
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.StandardFilterFactory"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> </analyzer>
> </fieldType>
>
> Relevant bits from solrconfig.xml:
> --------------------------------------
> <requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
> <lst name="defaults">
> <str name="suggest.dictionary">mySuggester</str>
> <str name="suggest">true</str>
> <str name="suggest.count">10</str>
> </lst>
> <arr name="components">
> <str>suggest</str>
> </arr>
> </requestHandler>
> ...
> <searchComponent name="suggest" class="solr.SuggestComponent">
> <lst name="suggester">
> <str name="name">mySuggester</str>
> <str name="lookupImpl">AnalyzingInfixLookupFactory</str>
> <str name="indexPath">suggester_infix_dir</str>
> <str name="dictionaryImpl">DocumentDictionaryFactory</str>
> <str name="field">text_suggest</str>
> <str name="suggestAnalyzerFieldType">string</str>
> <str name="buildOnStartup">false</str>
> <str name="buildOnCommit">false</str>
> </lst>
> </searchComponent>
>
> I build the dictionary like this:
>
> http://example.com:8983/solr/rrib/suggest?suggest.build=true
>
> and get back this response:
>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">964</int>
> </lst>
> <str name="command">build</str>
> </response>
>
> I then attempt a query like this:
>
> http://example.com:8983/solr/rrib/suggest?q=re
>
> and get back this response:
>
> <response>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">37</int>
> </lst>
> <lst name="suggest">
> <lst name="mySuggester">
> <lst name="re">
> <int name="numFound">0</int>
> <arr name="suggestions"/>
> </lst>
> </lst>
> </lst>
> </response>
>
> In Solr's Admin interface, I see the following under OTHER->suggest on
> Plugins/Stats page:
>
> class:suggest
> description:Suggester component
> src:
> version:6.1.0
> stats:
> mySuggester: SolrSuggester [ name=mySuggester,
> sourceLocation=null, storeDir=,
> lookupImpl=AnalyzingInfixLookupFactory,
> dictionaryImpl=DocumentDictionaryFactory, sizeInBytes=6795 ]
> totalSizeInBytes: 6795
>
> The value of 6,795 bytes seems pretty small to me for a repository of
> 403 XML files containing about 1.5 MB of mark-up. Perhaps that is a clue
> that the dictionary has not been fully populated, which probably
> explains the empty result sets, but I cannot figure out why
>
> Any assistance would be gratefully received.
>
> Thank you.
>
> On 5/6/2016 9:42 AM, Erick Erickson wrote:
> > First off, kudos for providing the details, that really helps!
> >
> > The root of your problem is that your suggest field has stored="false".
> > DocumentDictionaryFactory reads through all the
> > docs in your corpus, extracts the stored data and puts it in the FST. Since
> > you don't have any stored data your FST is...er...minimal.
> >
> > I'd also add
> > <str name="storeDir">suggester_fuzzy_dir</str>
> > to the searchComponent. You'll find the FST on disk in that directory where it
> > can be read next time Solr starts up. It is also helpful for figuring out
> > whether there are suggestions to be had.
> >
> > And a minor nit, you probably don't want to specify suggest.dictionary
> > in your query,
> > that's already specified in your config.
> >
> > And it looks like you're alive to the fact that with that setup
> > capitalization matters
> > as does the fact that these suggestions be matched from the beginning of the
> > field...
> >
> > Best,
> > Erick
> >
> > On Thu, May 5, 2016 at 1:05 AM, Grigoris Iliopoulos
> > <gr...@gmail.com> wrote:
> >> Hi there,
> >>
> >> I want to use the Solr suggester component for city names. I have the
> >> following settings:
> >> schema.xml
> >>
> >> Field definition
> >>
> >> <fieldType class="solr.TextField" name="textSuggest" positionIncrementGap="100">
> >> <analyzer>
> >> <tokenizer class="solr.StandardTokenizerFactory"/>
> >> <filter class="solr.StandardFilterFactory"/>
> >> <filter class="solr.LowerCaseFilterFactory"/>
> >> </analyzer>
> >> </fieldType>
> >>
> >> The field i want to apply the suggester on
> >>
> >> <field name="city" type="string" indexed="true" stored="false"/>
> >>
> >> The copy field
> >>
> >> <copyField source="city" dest="citySuggest"/>
> >>
> >> The field
> >>
> >> <field name="citySuggest" type="textSuggest" stored="false" indexed="true" />
> >>
> >> solr-config.xml
> >>
> >> <requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
> >> <lst name="defaults">
> >> <str name="suggest">true</str>
> >> <str name="suggest.count">10</str>
> >> <str name="suggest.dictionary">mySuggester</str>
> >> </lst>
> >> <arr name="components">
> >> <str>suggest</str>
> >> </arr>
> >> </requestHandler>
> >>
> >>
> >>
> >> <searchComponent name="suggest" class="solr.SuggestComponent">
> >> <lst name="suggester">
> >> <str name="name">mySuggester</str>
> >> <str name="lookupImpl">FuzzyLookupFactory</str>
> >> <str name="dictionaryImpl">DocumentDictionaryFactory</str>
> >> <str name="field">citySuggest</str>
> >> <str name="suggestAnalyzerFieldType">string</str>
> >> </lst>
> >> </searchComponent>
> >>
> >> Then i run
> >>
> >> http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath&suggest.build=true
> >>
> >> to build the suggest component
> >>
> >> Finally i run
> >>
> >> http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath
> >>
> >> but i get an empty result set
> >>
> >> {"responseHeader":{"status":0,"QTime":0},"suggest":{"mySuggester":{"Ath":{"numFound":0,"suggestions":[]}}}}
> >>
> >> Are there any obvious mistakes? Any thoughts?
> >
>
>
>
Re: Solr Suggester no results
Posted by Bradley Belyeu <br...@life.church>.
Scott, I’m fairly new to suggesters having just recently built my first one. But where my configuration differs from yours is on this line:
<str name="suggestAnalyzerFieldType">string</str>
I used the field type name that I had defined instead like:
<str name="suggestAnalyzerFieldType"> textSuggest </str>
I’m not certain that would help, but I can’t see where your config is much different form mine elsewhere.
What version of Solr are you running?
On 8/25/16, 6:41 PM, "Scott Vanderbilt" <li...@datagenic.com> wrote:
I'm having the exact same problem the O.P. describes from his email back
in May, but my configuration dose not have the same defect his had. So I
am at a loss to understand why my suggest queries are returning no results.
Here is my config:
Relevant bits from schema.xml:
--------------------------------------
<field name="roadName" type="string" indexed="true" stored="true"
multiValued="true"/>
<field name="commentaryP" type="text_general" indexed="true"
stored="true" multiValued="true"/>
...
<field name="text_suggest" type="textSuggest" indexed="true"
stored="true" multiValued="true" />
...
<copyField source="roadName" dest="text_suggest" />
<copyField source="commentaryP" dest="text_suggest" />
...
<fieldType class="solr.TextField" name="textSuggest"
positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
Relevant bits from solrconfig.xml:
--------------------------------------
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="suggest.dictionary">mySuggester</str>
<str name="suggest">true</str>
<str name="suggest.count">10</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
...
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">AnalyzingInfixLookupFactory</str>
<str name="indexPath">suggester_infix_dir</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">text_suggest</str>
<str name="suggestAnalyzerFieldType">string</str>
<str name="buildOnStartup">false</str>
<str name="buildOnCommit">false</str>
</lst>
</searchComponent>
I build the dictionary like this:
http://example.com:8983/solr/rrib/suggest?suggest.build=true
and get back this response:
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">964</int>
</lst>
<str name="command">build</str>
</response>
I then attempt a query like this:
http://example.com:8983/solr/rrib/suggest?q=re
and get back this response:
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">37</int>
</lst>
<lst name="suggest">
<lst name="mySuggester">
<lst name="re">
<int name="numFound">0</int>
<arr name="suggestions"/>
</lst>
</lst>
</lst>
</response>
In Solr's Admin interface, I see the following under OTHER->suggest on
Plugins/Stats page:
class:suggest
description:Suggester component
src:
version:6.1.0
stats:
mySuggester: SolrSuggester [ name=mySuggester,
sourceLocation=null, storeDir=,
lookupImpl=AnalyzingInfixLookupFactory,
dictionaryImpl=DocumentDictionaryFactory, sizeInBytes=6795 ]
totalSizeInBytes: 6795
The value of 6,795 bytes seems pretty small to me for a repository of
403 XML files containing about 1.5 MB of mark-up. Perhaps that is a clue
that the dictionary has not been fully populated, which probably
explains the empty result sets, but I cannot figure out why
Any assistance would be gratefully received.
Thank you.
On 5/6/2016 9:42 AM, Erick Erickson wrote:
> First off, kudos for providing the details, that really helps!
>
> The root of your problem is that your suggest field has stored="false".
> DocumentDictionaryFactory reads through all the
> docs in your corpus, extracts the stored data and puts it in the FST. Since
> you don't have any stored data your FST is...er...minimal.
>
> I'd also add
> <str name="storeDir">suggester_fuzzy_dir</str>
> to the searchComponent. You'll find the FST on disk in that directory where it
> can be read next time Solr starts up. It is also helpful for figuring out
> whether there are suggestions to be had.
>
> And a minor nit, you probably don't want to specify suggest.dictionary
> in your query,
> that's already specified in your config.
>
> And it looks like you're alive to the fact that with that setup
> capitalization matters
> as does the fact that these suggestions be matched from the beginning of the
> field...
>
> Best,
> Erick
>
> On Thu, May 5, 2016 at 1:05 AM, Grigoris Iliopoulos
> <gr...@gmail.com> wrote:
>> Hi there,
>>
>> I want to use the Solr suggester component for city names. I have the
>> following settings:
>> schema.xml
>>
>> Field definition
>>
>> <fieldType class="solr.TextField" name="textSuggest" positionIncrementGap="100">
>> <analyzer>
>> <tokenizer class="solr.StandardTokenizerFactory"/>
>> <filter class="solr.StandardFilterFactory"/>
>> <filter class="solr.LowerCaseFilterFactory"/>
>> </analyzer>
>> </fieldType>
>>
>> The field i want to apply the suggester on
>>
>> <field name="city" type="string" indexed="true" stored="false"/>
>>
>> The copy field
>>
>> <copyField source="city" dest="citySuggest"/>
>>
>> The field
>>
>> <field name="citySuggest" type="textSuggest" stored="false" indexed="true" />
>>
>> solr-config.xml
>>
>> <requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
>> <lst name="defaults">
>> <str name="suggest">true</str>
>> <str name="suggest.count">10</str>
>> <str name="suggest.dictionary">mySuggester</str>
>> </lst>
>> <arr name="components">
>> <str>suggest</str>
>> </arr>
>> </requestHandler>
>>
>>
>>
>> <searchComponent name="suggest" class="solr.SuggestComponent">
>> <lst name="suggester">
>> <str name="name">mySuggester</str>
>> <str name="lookupImpl">FuzzyLookupFactory</str>
>> <str name="dictionaryImpl">DocumentDictionaryFactory</str>
>> <str name="field">citySuggest</str>
>> <str name="suggestAnalyzerFieldType">string</str>
>> </lst>
>> </searchComponent>
>>
>> Then i run
>>
>> http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath&suggest.build=true
>>
>> to build the suggest component
>>
>> Finally i run
>>
>> http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath
>>
>> but i get an empty result set
>>
>> {"responseHeader":{"status":0,"QTime":0},"suggest":{"mySuggester":{"Ath":{"numFound":0,"suggestions":[]}}}}
>>
>> Are there any obvious mistakes? Any thoughts?
>