You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "O. Olson" <ol...@yahoo.it> on 2015/05/06 18:39:42 UTC
Trying to get AnalyzingInfixSuggester to work in Solr?
I'm trying to get the AnalyzingInfixSuggester to work but I'm not successful.
I'd be grateful if someone can point me to a working example.
Problem:
My content is product descriptions similar to a BestBuy or NewEgg catalog.
My problem is that I'm getting only single words in the suggester results.
E.g. if I type 'len', I get the suggester results like 'Lenovo' but not
'Lenovo laptop' or something larger/longer than a single word.
There is a suggestion here:
http://blog.mikemccandless.com/2013/06/a-new-lucene-suggester-based-on-infix.html
that the search at:
http://jirasearch.mikemccandless.com/search.py?index=jira is powered by the
AnalyzingInfixSuggester If this is true, when I use this suggester, I get
more than a few words in the suggester results, but I don't with my setup
i.e. on my setup I get only single words. My configuration is
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggest</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str
name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory</str>
<str name="field">text</str>
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
<str name="suggestAnalyzerFieldType">text_general</str>
<bool name="exactMatchFirst">true</bool>
</lst>
</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler"
name="/suggest">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggest</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
I copy the contents of all of my fields to a single field called 'text'. The
' text_general' type is exactly as in the solr examples:
http://svn.apache.org/viewvc/lucene/dev/trunk/solr/example/example-DIH/solr/db/conf/schema.xml?view=markup
I'd be grateful if anyone can help me. I don't know what to look at. Thank
you in adance.
O. O.
--
View this message in context: http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by "O. Olson" <ol...@yahoo.it>.
Thank you Rajesh for your persistence. I now got it to work. In my original
email/message, I mentioned that I use 'text_general' as defined in the
examples:
http://svn.apache.org/viewvc/lucene/dev/trunk/solr/example/example-DIH/solr/db/conf/schema.xml?view=markup
I'm sorry I did not mention this again later.
Your definition of 'text_general' is a lot different from what's in the
examples. However, once I used it, I got this to work just as you said.
Thank you,
O. O.
Rajesh Hazari wrote
> yes "textSuggest" is of type "text_general" with below definition
> <fieldType name="text_general" class="solr.TextField"
> positionIncrementGap="100" sortMissingLast="true" omitNorms="true">
>
> <analyzer type="index">
>
> <tokenizer class="solr.ClassicTokenizerFactory"/>
> <filter class="solr.ClassicFilterFactory"/>
> <filter class="solr.LowerCaseFilterFactory"/>
>
> <filter class="solr.KeywordMarkerFilterFactory"
> protected="protwords.txt"/>
> <filter class="solr.ShingleFilterFactory" maxShingleSize="5"
> outputUnigrams="true"/>
>
> </analyzer>
>
> <analyzer type="query">
> <charFilter class="solr.MappingCharFilterFactory"
> mapping="mapping-FoldToASCII.txt"/>
>
> <tokenizer class="solr.ClassicTokenizerFactory"/>
> <filter class="solr.ClassicFilterFactory"/>
>
> <filter class="solr.LowerCaseFilterFactory"/>
>
> <filter class="solr.KeywordMarkerFilterFactory"
> protected="protwords.txt"/>
> <filter class="solr.ShingleFilterFactory" maxShingleSize="5"
> outputUnigrams="true"/>
>
> </analyzer>
>
> </fieldType>
> *Rajesh.*
--
View this message in context: http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204334.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by Rajesh Hazari <ra...@gmail.com>.
yes "textSuggest" is of type "text_general" with below definition
<fieldType name="text_general" class="solr.TextField"
positionIncrementGap="100" sortMissingLast="true" omitNorms="true">
<analyzer type="index">
<tokenizer class="solr.ClassicTokenizerFactory"/>
<filter class="solr.ClassicFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory"
protected="protwords.txt"/>
<filter class="solr.ShingleFilterFactory" maxShingleSize="5"
outputUnigrams="true"/>
</analyzer>
<analyzer type="query">
<charFilter class="solr.MappingCharFilterFactory"
mapping="mapping-FoldToASCII.txt"/>
<tokenizer class="solr.ClassicTokenizerFactory"/>
<filter class="solr.ClassicFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory"
protected="protwords.txt"/>
<filter class="solr.ShingleFilterFactory" maxShingleSize="5"
outputUnigrams="true"/>
</analyzer>
</fieldType>
*Rajesh.*
On Wed, May 6, 2015 at 4:50 PM, O. Olson <ol...@yahoo.it> wrote:
> Thank you Rajesh for responding so quickly. I tried it again with a restart
> and a reimport and I still cannot get this to work i.e. I'm seeing no
> difference.
>
> I'm wondering how you define: 'textSuggest' in your schema? In my case I
> use
> the field 'text' that is defined as:
>
> <field name="text" type="text_general" indexed="true" stored="false"
> multiValued="true"/>
>
> I'm wondering if your 'textSuggest' is of type "text_general" ?
>
> Thank you again for your help
> O. O.
>
>
> Rajesh Hazari wrote
> > I just tested your config with my schema and it worked.
> >
> > my config :
> >
> > <searchComponent class="solr.SpellCheckComponent" name="suggest1">
> >
> > <lst name="spellchecker">
> >
> > <str name="name">
> > suggest
> > </str>
> >
> > <str name="classname">
> > org.apache.solr.spelling.suggest.Suggester
> > </str>
> >
> > <str
> > name="lookupImpl">
> > org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory
> > </str>
> >
> > <str name="field">
> > textSuggest
> > </str>
> >
> > <float name="threshold">
> > 0.005
> > </float>
> >
> > <str name="buildOnCommit">
> > true
> > </str>
> >
> > <str name="suggestAnalyzerFieldType">
> > text_general
> > </str>
> >
> > <bool name="exactMatchFirst">
> > true
> > </bool>
> >
> > </lst>
> >
> > </searchComponent>
> >
> > <queryConverter name="queryConverter"
> > class="org.apache.solr.spelling.SuggestQueryConverter"/>
> >
> > <requestHandler class="org.apache.solr.handler.component.SearchHandler"
> > name="/suggest1">
> >
> > <lst name="defaults">
> >
> > <str name="spellcheck">
> > true
> > </str>
> >
> > <str name="spellcheck.dictionary">
> > suggest
> > </str>
> >
> > <str name="spellcheck.onlyMorePopular">
> > true
> > </str>
> >
> > <str name="spellcheck.count">
> > 5
> > </str>
> >
> > <str name="spellcheck.collate">
> > true
> > </str>
> >
> > </lst>
> >
> > <arr name="components">
> >
> > <str>
> > suggest1
> > </str>
> >
> > </arr>
> >
> > </requestHandler>
> >
> >
> http://localhost:8585/solr/collection1/suggest1?q=apple&rows=10&wt=json&indent=true
> >
> > {
> > "responseHeader":{
> > "status":0,
> > "QTime":2},
> > "spellcheck":{
> > "suggestions":[
> > "apple",{
> > "numFound":5,
> > "startOffset":0,
> > "endOffset":5,
> > "suggestion":["
> *
> > apple
> *
> > ",
> > "
> *
> > apple
> *
> > and",
> > "
> *
> > apple
> *
> > and facebook",
> > "
> *
> > apple
> *
> > and facebook learn",
> > "
> *
> > apple
> *
> > and facebook learn from"]},
> > "collation","
> *
> > apple
> *
> > "]}}
> >
> >
> >
> > *Rajesh**.*
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204208.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
RE: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by "Reitzel, Charles" <Ch...@tiaa-cref.org>.
Fwiw, we ended up preferring the 4.x spellcheck approach. For starters, it is supported by SolrJ ... :-)
But more importantly, we wanted a mix of both terms and field values in our suggestions. We found the Suggester component doesn't do that. We also weren't interested in matching in the middle of words. Partial prefix matching was better and, thus, we used an ngram query.
In addition, we liked the Amazon style "xyz in Dept X, xyz in Dept Y" suggestions, which we used facets in combination with the ngram query to produce.
Finally, we needed to make a minor patch to get document frequency information about terms (and collations) provided by the SpellCheckComponent.
https://issues.apache.org/jira/browse/SOLR-7144
So, to summarize, we ended up with a 2-pass suggestion approach:
pass 1: spellcheck with document frequency and collation using WFSTLookupFactory and org.apache.solr.spelling.suggest.Suggester.
pass 2: if spellcheck has corrections?, use 1st correction instead of original term as query for against an ngram field (using copyTo to populate from fields we care about). This query also has a field facet. The facet values are used as "${queryTerm} in ${facet}" suggestions. Specified fields from matching docs are used as suggestions (like the suggester component).
Please don't take this to mean you should be doing anything like what we are doing. But, rather, I'm urging you to dig deeper into your suggestion functionality and think hard about what really makes sense for your application. It's a major usability issue for search apps.
-----Original Message-----
From: O. Olson [mailto:olson_ord@yahoo.it]
Sent: Thursday, May 07, 2015 4:19 PM
To: solr-user@lucene.apache.org
Subject: Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Thank you Erick. I'm sorry I did not mention this earlier, but I am still on Solr 4.10.3. Once I upgrade to Solr 5.0+ , I would consider your suggestion in your blog post.
O. O.
Erick Erickson wrote
> Uh, you mean because I forgot to pate in the URL? Siiggggh...
>
> Anyway, the URL is irrelevant now that you've solved your problem, but
> in case you're interested:
> http://lucidworks.com/blog/solr-suggester/
>
> Sorry for the confusion.
> Erick
--
View this message in context: http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204392.html
Sent from the Solr - User mailing list archive at Nabble.com.
*************************************************************************
This e-mail may contain confidential or privileged information.
If you are not the intended recipient, please notify the sender immediately and then delete it.
TIAA-CREF
*************************************************************************
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by "O. Olson" <ol...@yahoo.it>.
Thank you Erick. I'm sorry I did not mention this earlier, but I am still on
Solr 4.10.3. Once I upgrade to Solr 5.0+ , I would consider your suggestion
in your blog post.
O. O.
Erick Erickson wrote
> Uh, you mean because I forgot to pate in the URL? Siiggggh...
>
> Anyway, the URL is irrelevant now that you've solved your problem, but
> in case you're interested:
> http://lucidworks.com/blog/solr-suggester/
>
> Sorry for the confusion.
> Erick
--
View this message in context: http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204392.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by Erick Erickson <er...@gmail.com>.
Uh, you mean because I forgot to pate in the URL? Siiggggh...
Anyway, the URL is irrelevant now that you've solved your problem, but
in case you're interested:
http://lucidworks.com/blog/solr-suggester/
Sorry for the confusion.
Erick
On Thu, May 7, 2015 at 9:12 AM, Alessandro Benedetti
<be...@gmail.com> wrote:
> When working with Suggesters I "suggest" to take a deep look to this guide :
> http://lucidworks.com/blog/solr-suggester/
>
> It was really helpful.
>
> Cheers
>
> 2015-05-07 16:58 GMT+01:00 Rajesh Hazari <ra...@gmail.com>:
>
>> Good to know that its working as expected.
>>
>> I have some couple of question on your autosuggest implementation.
>>
>> I see that you are using SpellcheckComponent instead of SuggestComponent
>> are you using this intentionally if not plz read this
>> https://cwiki.apache.org/confluence/display/solr/Suggester
>>
>> I am working on an issue in suggester just sharing once again in this
>> community just in-case if you are any others out have this in their list.
>>
>>
>> http://stackoverflow.com/questions/27847707/solr-autosuggest-to-stop-filter-suggesting-the-phrase-that-ends-with-stopwords
>>
>> *thanks,*
>> *Rajesh**.*
>>
>> On Thu, May 7, 2015 at 11:26 AM, O. Olson <ol...@yahoo.it> wrote:
>>
>> > Thank you Erick. I have no clue what you are referring to when you used
>> to
>> > word "this"? Are you referring to my question in my original
>> > email/message?
>> >
>> >
>> > Erick Erickson wrote
>> > > Have you seen this? I tried to make something end-to-end with assorted
>> > > "gotchas" identified....
>> > >
>> > > Best,
>> > > Erick
>> >
>> >
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> >
>> http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204336.html
>> > Sent from the Solr - User mailing list archive at Nabble.com.
>> >
>>
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by Alessandro Benedetti <be...@gmail.com>.
When working with Suggesters I "suggest" to take a deep look to this guide :
http://lucidworks.com/blog/solr-suggester/
It was really helpful.
Cheers
2015-05-07 16:58 GMT+01:00 Rajesh Hazari <ra...@gmail.com>:
> Good to know that its working as expected.
>
> I have some couple of question on your autosuggest implementation.
>
> I see that you are using SpellcheckComponent instead of SuggestComponent
> are you using this intentionally if not plz read this
> https://cwiki.apache.org/confluence/display/solr/Suggester
>
> I am working on an issue in suggester just sharing once again in this
> community just in-case if you are any others out have this in their list.
>
>
> http://stackoverflow.com/questions/27847707/solr-autosuggest-to-stop-filter-suggesting-the-phrase-that-ends-with-stopwords
>
> *thanks,*
> *Rajesh**.*
>
> On Thu, May 7, 2015 at 11:26 AM, O. Olson <ol...@yahoo.it> wrote:
>
> > Thank you Erick. I have no clue what you are referring to when you used
> to
> > word "this"? Are you referring to my question in my original
> > email/message?
> >
> >
> > Erick Erickson wrote
> > > Have you seen this? I tried to make something end-to-end with assorted
> > > "gotchas" identified....
> > >
> > > Best,
> > > Erick
> >
> >
> >
> >
> >
> > --
> > View this message in context:
> >
> http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204336.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
>
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by "O. Olson" <ol...@yahoo.it>.
Thank you Rajesh, Alessandro and Erick. I apparently did not have much
knowledge about the Suggester - in fact I had no clue that there is a
difference between the SpellcheckComponent and the SuggestComponent.
I would be reading about this, esp. Erick blog post on Lucidworks.
O. O.
Rajesh Hazari wrote
> Good to know that its working as expected.
>
> I have some couple of question on your autosuggest implementation.
>
> I see that you are using SpellcheckComponent instead of SuggestComponent
> are you using this intentionally if not plz read this
> https://cwiki.apache.org/confluence/display/solr/Suggester
>
> I am working on an issue in suggester just sharing once again in this
> community just in-case if you are any others out have this in their list.
>
> http://stackoverflow.com/questions/27847707/solr-autosuggest-to-stop-filter-suggesting-the-phrase-that-ends-with-stopwords
>
> *thanks,*
> *Rajesh**.*
--
View this message in context: http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204356.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by Rajesh Hazari <ra...@gmail.com>.
Good to know that its working as expected.
I have some couple of question on your autosuggest implementation.
I see that you are using SpellcheckComponent instead of SuggestComponent
are you using this intentionally if not plz read this
https://cwiki.apache.org/confluence/display/solr/Suggester
I am working on an issue in suggester just sharing once again in this
community just in-case if you are any others out have this in their list.
http://stackoverflow.com/questions/27847707/solr-autosuggest-to-stop-filter-suggesting-the-phrase-that-ends-with-stopwords
*thanks,*
*Rajesh**.*
On Thu, May 7, 2015 at 11:26 AM, O. Olson <ol...@yahoo.it> wrote:
> Thank you Erick. I have no clue what you are referring to when you used to
> word "this"? Are you referring to my question in my original
> email/message?
>
>
> Erick Erickson wrote
> > Have you seen this? I tried to make something end-to-end with assorted
> > "gotchas" identified....
> >
> > Best,
> > Erick
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204336.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by "O. Olson" <ol...@yahoo.it>.
Thank you Erick. I have no clue what you are referring to when you used to
word "this"? Are you referring to my question in my original email/message?
Erick Erickson wrote
> Have you seen this? I tried to make something end-to-end with assorted
> "gotchas" identified....
>
> Best,
> Erick
--
View this message in context: http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204336.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by Erick Erickson <er...@gmail.com>.
Have you seen this? I tried to make something end-to-end with assorted
"gotchas" identified....
Best,
Erick
On Wed, May 6, 2015 at 3:09 PM, O. Olson <ol...@yahoo.it> wrote:
> Thank you Rajesh. I think I got a bit of help from the answer at:
> http://stackoverflow.com/a/29743945
>
> While that example sort of worked for me, I'm not had the time to test what
> works and what didn't.
>
> So far I have found that I need the the field in my searchComponent to be of
> type 'string'. In my original example I had this as text_general. Next I
> used the suggest_string fieldType as defined in the StackOverflow answer. I
> also removed your queryConverter, and it still works, so I think it's not
> needed.
>
> Thank you very much,
> O. O.
>
>
>
> Rajesh Hazari wrote
>> I just tested your config with my schema and it worked.
>>
>> my config :
>>
>> <searchComponent class="solr.SpellCheckComponent" name="suggest1">
>>
>> <lst name="spellchecker">
>>
>> <str name="name">
>> suggest
>> </str>
>>
>> <str name="classname">
>> org.apache.solr.spelling.suggest.Suggester
>> </str>
>>
>> <str
>> name="lookupImpl">
>> org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory
>> </str>
>>
>> <str name="field">
>> textSuggest
>> </str>
>>
>> <float name="threshold">
>> 0.005
>> </float>
>>
>> <str name="buildOnCommit">
>> true
>> </str>
>>
>> <str name="suggestAnalyzerFieldType">
>> text_general
>> </str>
>>
>> <bool name="exactMatchFirst">
>> true
>> </bool>
>>
>> </lst>
>>
>> </searchComponent>
>>
>> <queryConverter name="queryConverter"
>> class="org.apache.solr.spelling.SuggestQueryConverter"/>
>>
>> <requestHandler class="org.apache.solr.handler.component.SearchHandler"
>> name="/suggest1">
>>
>> <lst name="defaults">
>>
>> <str name="spellcheck">
>> true
>> </str>
>>
>> <str name="spellcheck.dictionary">
>> suggest
>> </str>
>>
>> <str name="spellcheck.onlyMorePopular">
>> true
>> </str>
>>
>> <str name="spellcheck.count">
>> 5
>> </str>
>>
>> <str name="spellcheck.collate">
>> true
>> </str>
>>
>> </lst>
>>
>> <arr name="components">
>>
>> <str>
>> suggest1
>> </str>
>>
>> </arr>
>>
>> </requestHandler>
>>
>> http://localhost:8585/solr/collection1/suggest1?q=apple&rows=10&wt=json&indent=true
>>
>> {
>> "responseHeader":{
>> "status":0,
>> "QTime":2},
>> "spellcheck":{
>> "suggestions":[
>> "apple",{
>> "numFound":5,
>> "startOffset":0,
>> "endOffset":5,
>> "suggestion":["
> *
>> apple
> *
>> ",
>> "
> *
>> apple
> *
>> and",
>> "
> *
>> apple
> *
>> and facebook",
>> "
> *
>> apple
> *
>> and facebook learn",
>> "
> *
>> apple
> *
>> and facebook learn from"]},
>> "collation","
> *
>> apple
> *
>> "]}}
>>
>>
>>
>> *Rajesh**.*
>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204222.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by "O. Olson" <ol...@yahoo.it>.
Thank you Rajesh. I think I got a bit of help from the answer at:
http://stackoverflow.com/a/29743945
While that example sort of worked for me, I'm not had the time to test what
works and what didn't.
So far I have found that I need the the field in my searchComponent to be of
type 'string'. In my original example I had this as text_general. Next I
used the suggest_string fieldType as defined in the StackOverflow answer. I
also removed your queryConverter, and it still works, so I think it's not
needed.
Thank you very much,
O. O.
Rajesh Hazari wrote
> I just tested your config with my schema and it worked.
>
> my config :
>
> <searchComponent class="solr.SpellCheckComponent" name="suggest1">
>
> <lst name="spellchecker">
>
> <str name="name">
> suggest
> </str>
>
> <str name="classname">
> org.apache.solr.spelling.suggest.Suggester
> </str>
>
> <str
> name="lookupImpl">
> org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory
> </str>
>
> <str name="field">
> textSuggest
> </str>
>
> <float name="threshold">
> 0.005
> </float>
>
> <str name="buildOnCommit">
> true
> </str>
>
> <str name="suggestAnalyzerFieldType">
> text_general
> </str>
>
> <bool name="exactMatchFirst">
> true
> </bool>
>
> </lst>
>
> </searchComponent>
>
> <queryConverter name="queryConverter"
> class="org.apache.solr.spelling.SuggestQueryConverter"/>
>
> <requestHandler class="org.apache.solr.handler.component.SearchHandler"
> name="/suggest1">
>
> <lst name="defaults">
>
> <str name="spellcheck">
> true
> </str>
>
> <str name="spellcheck.dictionary">
> suggest
> </str>
>
> <str name="spellcheck.onlyMorePopular">
> true
> </str>
>
> <str name="spellcheck.count">
> 5
> </str>
>
> <str name="spellcheck.collate">
> true
> </str>
>
> </lst>
>
> <arr name="components">
>
> <str>
> suggest1
> </str>
>
> </arr>
>
> </requestHandler>
>
> http://localhost:8585/solr/collection1/suggest1?q=apple&rows=10&wt=json&indent=true
>
> {
> "responseHeader":{
> "status":0,
> "QTime":2},
> "spellcheck":{
> "suggestions":[
> "apple",{
> "numFound":5,
> "startOffset":0,
> "endOffset":5,
> "suggestion":["
*
> apple
*
> ",
> "
*
> apple
*
> and",
> "
*
> apple
*
> and facebook",
> "
*
> apple
*
> and facebook learn",
> "
*
> apple
*
> and facebook learn from"]},
> "collation","
*
> apple
*
> "]}}
>
>
>
> *Rajesh**.*
--
View this message in context: http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204222.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by "O. Olson" <ol...@yahoo.it>.
Thank you Rajesh for responding so quickly. I tried it again with a restart
and a reimport and I still cannot get this to work i.e. I'm seeing no
difference.
I'm wondering how you define: 'textSuggest' in your schema? In my case I use
the field 'text' that is defined as:
<field name="text" type="text_general" indexed="true" stored="false"
multiValued="true"/>
I'm wondering if your 'textSuggest' is of type "text_general" ?
Thank you again for your help
O. O.
Rajesh Hazari wrote
> I just tested your config with my schema and it worked.
>
> my config :
>
> <searchComponent class="solr.SpellCheckComponent" name="suggest1">
>
> <lst name="spellchecker">
>
> <str name="name">
> suggest
> </str>
>
> <str name="classname">
> org.apache.solr.spelling.suggest.Suggester
> </str>
>
> <str
> name="lookupImpl">
> org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory
> </str>
>
> <str name="field">
> textSuggest
> </str>
>
> <float name="threshold">
> 0.005
> </float>
>
> <str name="buildOnCommit">
> true
> </str>
>
> <str name="suggestAnalyzerFieldType">
> text_general
> </str>
>
> <bool name="exactMatchFirst">
> true
> </bool>
>
> </lst>
>
> </searchComponent>
>
> <queryConverter name="queryConverter"
> class="org.apache.solr.spelling.SuggestQueryConverter"/>
>
> <requestHandler class="org.apache.solr.handler.component.SearchHandler"
> name="/suggest1">
>
> <lst name="defaults">
>
> <str name="spellcheck">
> true
> </str>
>
> <str name="spellcheck.dictionary">
> suggest
> </str>
>
> <str name="spellcheck.onlyMorePopular">
> true
> </str>
>
> <str name="spellcheck.count">
> 5
> </str>
>
> <str name="spellcheck.collate">
> true
> </str>
>
> </lst>
>
> <arr name="components">
>
> <str>
> suggest1
> </str>
>
> </arr>
>
> </requestHandler>
>
> http://localhost:8585/solr/collection1/suggest1?q=apple&rows=10&wt=json&indent=true
>
> {
> "responseHeader":{
> "status":0,
> "QTime":2},
> "spellcheck":{
> "suggestions":[
> "apple",{
> "numFound":5,
> "startOffset":0,
> "endOffset":5,
> "suggestion":["
*
> apple
*
> ",
> "
*
> apple
*
> and",
> "
*
> apple
*
> and facebook",
> "
*
> apple
*
> and facebook learn",
> "
*
> apple
*
> and facebook learn from"]},
> "collation","
*
> apple
*
> "]}}
>
>
>
> *Rajesh**.*
--
View this message in context: http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204208.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by Rajesh Hazari <ra...@gmail.com>.
I just tested your config with my schema and it worked.
my config :
<searchComponent class="solr.SpellCheckComponent" name="suggest1">
<lst name="spellchecker">
<str name="name">suggest</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str
name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory</str>
<str name="field">textSuggest</str>
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
<str name="suggestAnalyzerFieldType">text_general</str>
<bool name="exactMatchFirst">true</bool>
</lst>
</searchComponent>
<queryConverter name="queryConverter"
class="org.apache.solr.spelling.SuggestQueryConverter"/>
<requestHandler class="org.apache.solr.handler.component.SearchHandler"
name="/suggest1">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggest</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="components">
<str>suggest1</str>
</arr>
</requestHandler>
http://localhost:8585/solr/collection1/suggest1?q=apple&rows=10&wt=json&indent=true
{
"responseHeader":{
"status":0,
"QTime":2},
"spellcheck":{
"suggestions":[
"apple",{
"numFound":5,
"startOffset":0,
"endOffset":5,
"suggestion":["<b>apple</b>",
"<b>apple</b> and",
"<b>apple</b> and facebook",
"<b>apple</b> and facebook learn",
"<b>apple</b> and facebook learn from"]},
"collation","<b>apple</b>"]}}
*Rajesh**.*
On Wed, May 6, 2015 at 2:48 PM, Rajesh Hazari <ra...@gmail.com>
wrote:
> Just add the queryConverter definition in your solr config you should use
> see multiple term suggestions.
> and also make sure you have shingleFilterFactory as one of the filter in
> you schema field definitions for your field "text_general".
>
> <filter class="solr.ShingleFilterFactory" maxShingleSize="5"
> outputUnigrams="true"/>
>
>
> *Rajesh**.*
>
> On Wed, May 6, 2015 at 1:47 PM, O. Olson <ol...@yahoo.it> wrote:
>
>> Thank you Rajesh. I'm not familiar with the queryConverter. How do you
>> wire
>> it up to the rest of the setup? Right now, I just put it between the
>> SpellCheckComponent and the RequestHandler i.e. my config is as:
>>
>> <searchComponent class="solr.SpellCheckComponent" name="suggest">
>> <lst name="spellchecker">
>> <str name="name">suggest</str>
>> <str
>> name="classname">org.apache.solr.spelling.suggest.Suggester</str>
>> <str
>>
>> name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory</str>
>> <str name="field">text</str>
>> <float name="threshold">0.005</float>
>> <str name="buildOnCommit">true</str>
>> <str name="suggestAnalyzerFieldType">text_general</str>
>> <bool name="exactMatchFirst">true</bool>
>> </lst>
>> </searchComponent>
>>
>> <queryConverter name="queryConverter"
>> class="org.apache.solr.spelling.SuggestQueryConverter"/>
>>
>> <requestHandler class="org.apache.solr.handler.component.SearchHandler"
>> name="/suggest">
>> <lst name="defaults">
>> <str name="spellcheck">true</str>
>> <str name="spellcheck.dictionary">suggest</str>
>> <str name="spellcheck.onlyMorePopular">true</str>
>> <str name="spellcheck.count">5</str>
>> <str name="spellcheck.collate">true</str>
>> </lst>
>> <arr name="components">
>> <str>suggest</str>
>> </arr>
>> </requestHandler>
>>
>> Is this correct? I do not see any difference in my results i.e.
>> the
>> suggestions are the same as before.
>> O. O.
>>
>>
>>
>>
>>
>> Rajesh Hazari wrote
>> > make sure you have this query converter defined in your config
>> > <queryConverter name="queryConverter"
>> > class="org.apache.solr.spelling.SuggestQueryConverter"/>
>> > *Thanks,*
>> > *Rajesh**.*
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204173.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>
>
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by Rajesh Hazari <ra...@gmail.com>.
Just add the queryConverter definition in your solr config you should use
see multiple term suggestions.
and also make sure you have shingleFilterFactory as one of the filter in
you schema field definitions for your field "text_general".
<filter class="solr.ShingleFilterFactory" maxShingleSize="5"
outputUnigrams="true"/>
*Rajesh**.*
On Wed, May 6, 2015 at 1:47 PM, O. Olson <ol...@yahoo.it> wrote:
> Thank you Rajesh. I'm not familiar with the queryConverter. How do you wire
> it up to the rest of the setup? Right now, I just put it between the
> SpellCheckComponent and the RequestHandler i.e. my config is as:
>
> <searchComponent class="solr.SpellCheckComponent" name="suggest">
> <lst name="spellchecker">
> <str name="name">suggest</str>
> <str
> name="classname">org.apache.solr.spelling.suggest.Suggester</str>
> <str
>
> name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory</str>
> <str name="field">text</str>
> <float name="threshold">0.005</float>
> <str name="buildOnCommit">true</str>
> <str name="suggestAnalyzerFieldType">text_general</str>
> <bool name="exactMatchFirst">true</bool>
> </lst>
> </searchComponent>
>
> <queryConverter name="queryConverter"
> class="org.apache.solr.spelling.SuggestQueryConverter"/>
>
> <requestHandler class="org.apache.solr.handler.component.SearchHandler"
> name="/suggest">
> <lst name="defaults">
> <str name="spellcheck">true</str>
> <str name="spellcheck.dictionary">suggest</str>
> <str name="spellcheck.onlyMorePopular">true</str>
> <str name="spellcheck.count">5</str>
> <str name="spellcheck.collate">true</str>
> </lst>
> <arr name="components">
> <str>suggest</str>
> </arr>
> </requestHandler>
>
> Is this correct? I do not see any difference in my results i.e. the
> suggestions are the same as before.
> O. O.
>
>
>
>
>
> Rajesh Hazari wrote
> > make sure you have this query converter defined in your config
> > <queryConverter name="queryConverter"
> > class="org.apache.solr.spelling.SuggestQueryConverter"/>
> > *Thanks,*
> > *Rajesh**.*
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204173.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by "O. Olson" <ol...@yahoo.it>.
Thank you Rajesh. I'm not familiar with the queryConverter. How do you wire
it up to the rest of the setup? Right now, I just put it between the
SpellCheckComponent and the RequestHandler i.e. my config is as:
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggest</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str
name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory</str>
<str name="field">text</str>
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
<str name="suggestAnalyzerFieldType">text_general</str>
<bool name="exactMatchFirst">true</bool>
</lst>
</searchComponent>
<queryConverter name="queryConverter"
class="org.apache.solr.spelling.SuggestQueryConverter"/>
<requestHandler class="org.apache.solr.handler.component.SearchHandler"
name="/suggest">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggest</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
Is this correct? I do not see any difference in my results i.e. the
suggestions are the same as before.
O. O.
Rajesh Hazari wrote
> make sure you have this query converter defined in your config
> <queryConverter name="queryConverter"
> class="org.apache.solr.spelling.SuggestQueryConverter"/>
> *Thanks,*
> *Rajesh**.*
--
View this message in context: http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163p4204173.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Trying to get AnalyzingInfixSuggester to work in Solr?
Posted by Rajesh Hazari <ra...@gmail.com>.
make sure you have this query converter defined in your config
<queryConverter name="queryConverter"
class="org.apache.solr.spelling.SuggestQueryConverter"/>
*Thanks,*
*Rajesh**.*
On Wed, May 6, 2015 at 12:39 PM, O. Olson <ol...@yahoo.it> wrote:
> I'm trying to get the AnalyzingInfixSuggester to work but I'm not
> successful.
> I'd be grateful if someone can point me to a working example.
>
> Problem:
> My content is product descriptions similar to a BestBuy or NewEgg catalog.
> My problem is that I'm getting only single words in the suggester results.
> E.g. if I type 'len', I get the suggester results like 'Lenovo' but not
> 'Lenovo laptop' or something larger/longer than a single word.
>
> There is a suggestion here:
>
> http://blog.mikemccandless.com/2013/06/a-new-lucene-suggester-based-on-infix.html
> that the search at:
> http://jirasearch.mikemccandless.com/search.py?index=jira is powered by
> the
> AnalyzingInfixSuggester If this is true, when I use this suggester, I get
> more than a few words in the suggester results, but I don't with my setup
> i.e. on my setup I get only single words. My configuration is
>
>
> <searchComponent class="solr.SpellCheckComponent" name="suggest">
> <lst name="spellchecker">
> <str name="name">suggest</str>
> <str
> name="classname">org.apache.solr.spelling.suggest.Suggester</str>
> <str
>
> name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory</str>
> <str name="field">text</str>
> <float name="threshold">0.005</float>
> <str name="buildOnCommit">true</str>
> <str name="suggestAnalyzerFieldType">text_general</str>
> <bool name="exactMatchFirst">true</bool>
> </lst>
> </searchComponent>
>
> <requestHandler class="org.apache.solr.handler.component.SearchHandler"
> name="/suggest">
> <lst name="defaults">
> <str name="spellcheck">true</str>
> <str name="spellcheck.dictionary">suggest</str>
> <str name="spellcheck.onlyMorePopular">true</str>
> <str name="spellcheck.count">5</str>
> <str name="spellcheck.collate">true</str>
> </lst>
> <arr name="components">
> <str>suggest</str>
> </arr>
> </requestHandler>
>
> I copy the contents of all of my fields to a single field called 'text'.
> The
> ' text_general' type is exactly as in the solr examples:
>
> http://svn.apache.org/viewvc/lucene/dev/trunk/solr/example/example-DIH/solr/db/conf/schema.xml?view=markup
>
> I'd be grateful if anyone can help me. I don't know what to look at. Thank
> you in adance.
>
> O. O.
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Trying-to-get-AnalyzingInfixSuggester-to-work-in-Solr-tp4204163.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>